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Abstract 


The  Spirit  River  School  Division  Educational  Quality  Indicators  Project  involved  the  development 
and  validation  of  a set  of  indicators  to  be  used  to  assess  teacher  effectiveness  in  a formative 
process.  We  believe  that  teachers  do  make  a difference,  for  as  Broudy  (1975)  stated:  ’’The  teacher 
is  still  the  key  to  schooling:  with  good  teaching,  almost  any  curriculum,  school  organization,  and 
administrative  invention  seems  to  succeed”  (p.  64).  Experience  and  the  literature  clearly  suggest 
that  teacher  and  administrator  involvement  in  the  selection  of  educational  indicators  was  critical  to 
the  success  of  a project  of  this  magnitude.  For  any  project  involving  key  components  of  action 
research,  it  was  necessary  to  gain  acceptance,  involvement  and  the  positive  initiative  of  staff.  The 
project  focused  on  professional  development  and  staff  inservice.  A collegial,  non-threatening 
model  for  more  effective  teaching  and  benefit  to  students  using  formative  supervision  and 
observation  was  developed  and  implemented  over  three  years.  It  focused  the  attention  of  the 
system  on  the  wider  range  of  student  educational  outcomes,  affective  and  behavioral  as  well  as 
cognitive. 
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Chapter  I 


INTRODUCTION 

Background 

In  December  1988,  the  Spirit  River  Schcx)l  Division  put  forward  a proposal  of  a pilot  project 
to  identify  educational  quality  indicators.  From  the  beginning,  this  jurisdiction  concentrated 
on  isolating  and  verifying  established  criteria  for  policies  on  teacher/staff/school  and  subject 
evaluations.  It  was  hoped  that  in  the  context  of  this  rural  jurisdiction,  criteria  and  processes 
could  be  refined  so  the  practical  results  could  be  shared  with  jurisdictions  of  similar  type,  and 
the  whole  province,  if  applicable. 

The  project  focused  on  identifying  criteria  for  teacher  effectiveness.  Experience  and  an  initial 
literature  review  indicated  that  for  any  project  involving  our  key  components  to  succeed,  it  was 
essential  to  gain  the  acceptance  and  involvement  of  teaching  staff.  The  response  from  staff  was 
heartening  and  positive.  A local  committee  of  eight  teachers  and  two  central  office  administrators 
reviewed  pertinent  research  and  literature.  They  developed  four  grouped  sets  of  indicators  of 
effective  teaching,  formulated  key  variables  which  could  impact  on  the  effectiveness  of 
implementation  and  articulated  clear  explanations  of  each  objective.  Data  gathering  instmments 
and  an  action  plan  were  developed  which  addressed  consequent  logistics  and  time  lines  for 
voluntary  implementation  in  all  division  schools. 

Context 

Spirit  River  School  Division  No.  47  is  located  in  northwestern  Alberta.  Its  western  boundary 
is  the  British  Columbia  border  and  on  the  east  side  it  is  confined  by  the  Smoky  River,  a total 
span  of  approximately  180  kilometers.  It  is  a relatively  sparsely  populated  area  of  approximately 
10,000  square  kilometres  with  agriculture  as  the  main  economic  activity.  The  Board  operates 
eight  schools  with  a total  enrolment  of  approximately  1,250  students.  Grade  configurations 
within  schools  vary,  ranging  from  one  elementary  school  to  two  K-12  schools.  Three  of  the 
schools  offer  senior  high  school  programs. 
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Rationale 


We  believe  that  teachers  do  make  a difference,  for  as  Broudy  (1975)  stated:  ’’The  teacher  is 
still  the  key  to  schooling:  with  good  teaching,  almost  any  curriculum,  school  organization, 
and  administrative  invention  seems  to  succeed”  (p.  64).  Experience  and  the  literature  clearly 
suggest  that  teacher  and  administrator  involvement  in  the  selection  of  educational  indicators 
was  critical  to  the  success  of  a project  of  this  magnitude.  For  any  project  involving  key 
components  of  action  research,  it  was  necessary  to  gain  acceptance,  involvement  and  the 
positive  initiative  of  staff.  The  project  focused  on  professional  development  and  staff  inservice. 
A collegial,  nonthreatening  model  for  more  effective  teaching  and  benefit  to  students  using 
formative  supervision  and  observation  was  developed  and  implemented  over  three  years. 

It  focused  the  attention  of  the  system  on  the  wider  range  of  student  educational  outcomes, 
affective  and  behavioral  as  well  as  cognitive. 


Purpose 

A wealth  of  information  on  effective  teaching  behaviors,  substantiated  by  practical  research,  has 
emerged  over  the  last  few  decades.  The  Spirit  River  School  Division  Indicators  Project  involved 
identification  and  validation  of  a set  of  indicators  that  can  be  used  to  assess  teacher  effectiveness. 
The  criteria  by  which  to  measure  teacher  effectiveness  are  of  central  importance  to  the  concept 
of  teacher  evaluation  and  effectiveness.  Normed  measures  of  teacher  behaviors  have  been 
established,  and  student  affective  and  cognitive  outcomes  as  they  relate  to  those  behaviors  are 
being  measured. 

The  question  posed  was:  Do  the  26  teaching  behaviors  we  have  identified  as  being  critical  to 
effective  teaching  have  an  impact  on  student  outcomes  in  both  the  cognitive  and  affective 
domains? 


Assumptions 

The  nature  of  this  project  was  such  that  there  was  considerable  reliance  upon  subjective  data. 
Student  attitude  surveys,  parent  surveys  and  teacher  performance  ratings  all  yielded  subjective 
data.  It  was  assumed  that  the  survey  instruments  yielded  information  which  was  an  accurate 
reflection  of  the  perceptions  of  students  and  parents.  The  establishment  of  a teacher  performance 
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baseline  required  an  assessment  of  teachers'  performance.  To  increase  the  consistency  of  teacher 
observations  and  to  define  different  levels  of  teacher  effectiveness,  "teacher  observers"  participated 
in  a three  hour  training  process. 


Definition 

The  EQI  committee  adopted  Oakes'  definition  of  an  educational  indicator, 

A statistic  about  the  educational  system,  that  reveals  something  about  its 
performance  or  health.  An  indicator  has  a reference  point  The  reference 
point  may  be  a previous  value,  the  value  of  a comparison  group,  or  some 
socially  determined  standard.  An  indicator  should  provide  at  least 
one  of  the  following: 

• information  about  the  education  system's  performance 

• features  of  the  system  known  to  be  linked  to  desired  outcomes 

• central  features  of  the  system 

• potential  or  existing  problem  area 

• information  that  is  policy  relevant  (1986,  pp.  1-2) 


Design 

This  project  concentrated  on  isolating  and  verifying  established  criteria  for  the  various  policies 
on  teacher,  school  and  subject  evaluations  which  were  already  in  place.  Ultimately,  the  project 
involved  developing  and  validating  a set  of  indicators  which  could  be  used  to  assess  teacher 
effectiveness  in  a formative  process. 

Voluntary  support  through  acceptance  and  involvement  was  received  from  staff.  A survey  of 
all  staff,  which  asked  them  to  indicate  the  criteria  each  felt  were  critical  to  improving  teacher 
effectiveness,  received  a response  in  excess  of  80  percent.  Eight  experienced  and  respected 
teachers  expressed  a keen  interest  in  serving  on  the  local  committee  whose  purpose  was  to 
finalize  the  indicators  and  establish  the  process  and  terms  of  reference  for  the  project  within 
the  established  guidelines. 
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This  committee  reviewed  all  pertinent  research  and  literature  and  developed  a set  of  26  criteria 
of  effective  teaching  and  formulated  key  variables  which  could  impact  the  effectiveness  of 
implementation  in  all  division  schools  on  a voluntary  basis.  The  committee  reviewed  the 
input  from  staff  and  finalized  the  working  project  documents.  Table  1 presents  the  26  criteria 
in  four  categories. 

During  the  first  year,  on  a voluntary  basis  and  in  a formative  process,  all  staff  had  the  opportunity 
to  meet  and  discuss  the  previously  developed  indicators  of  effective  teaching  or  visit  each  other’s 
classrooms  to  observe  one  or  more  of  these  specific,  established  areas.  Data  were  collected  on 
staff  participation,  specific  objectives  related  to  each  type  of  indicator,  local  or  out  of  school 
contacts  and  years  of  teaching  experience.  Information  was  also  gathered  on  geographic  distance, 
use  of  extra  time  provided  in  staffing,  staff  attitudes,  and  extended  professional  development 
activities.  Two  survey  instruments  were  developed  and  validated  using  Good's  (1963)  validation 
criteria  and  parents  and  students  were  subsequently  surveyed  to  gain  their  perspectives  of 
indicators  of  quality  teaching.  Surveys  were  distributed  to  all  students  in  grades  9, 10, 1 1 
and  12  and  to  parents  of  students  in  grades  2,  5,  8 and  11.  The  intent  of  these  representative 
surveys  was  to  gain  parent  and  student  input  into  the  process  and  criteria,  while  obtaining  their 
perception  on  validity  and/or  development  of  new  criteria.  A teacher  survey  on  satisfaction  and 
professional  development  involvement  was  also  conducted.  The  data  gathered  at  the  conclusion 
of  the  first  year  of  the  project  were  collated  and  presented  on  a system  level,  thereby  assuring  the 
anonymity  of  participants. 

In  the  second  year  of  the  project,  data  collection  was  expanded.  A decision  was  made  to  enhance 
the  credibility  of  the  project  as  a research  endeavor  through  the  establishment  of  a teacher 
performance  baseline  to  which  comparisons  could  be  made  at  a later  point  in  time.  Teaching 
behaviors  with  the  lowest  ratings  were  addressed  at  the  system  level  through  a variety  of 
professional  development  activities. 

A need  was  also  recognized  for  additional  student  outcome  measures  relevant  to  the  intent  of  the 
indicator  system.  The  Educational  Quality  Indicators:  Inventory  of  Assessment  Instruments 
(Alberta  Education,  1990)  was  consulted.  Sample  copies  of  a number  of  assessment  instruments 
were  reviewed  and  the  Classroom  Environment  Scale  (Moos  & Trickett,  1987)  was  selected  for 
grades  5,  8,  and  1 1 and  the  Student's  Perception  of  Ability  Scale  (Boersma  & Chapman,  1977) 
was  selected  for  grade  2 to  measure  student  outcomes  in  the  affective  domain.  The  former  served 
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to  supplement  data  gained  on  cognitive  outcomes  measured  by  provincial  diploma  exams  and 
achievement  tests.  The  data  collected  in  the  third  year  was  identical  to  that  of  the  second  year  of 
the  project 


Delimitations 

This  study  was  concerned  with  perceptions  in  one  rural  school  jurisdiction,  unique  in  many 
respects,  including  a large  geographical  area  and  a relatively  low  student  population.  The  findings 
may  not  apply  to  other  districts  in  the  province  whose  characteristics  and  circumstances  may  be 
considerably  different 

Only  aggregate  data  for  teachers  are  reported.  All  findings  apply  to  the  system  as  a whole  and 
not  to  any  one  school  or  any  one  identifiable  group  of  teachers. 


Limitations 

A limitation  of  the  project  is  the  accuracy  with  which  respondents  interpreted  the  survey  questions. 
Although  every  effort  was  made  to  eliminate  item  ambiguity,  potential  for  misinterpretation 
remained. 

Since  random  sampling  was  not  employed  in  the  selection  of  survey  candidates,  the  results  are 
not  necessarily  generalizable  to  other  jurisdictions. 


Overview 

Chapter  I provides  the  introduction  to  the  project,  identifies  its  purpose,  and  briefly  outlines  the 
context  within  which  it  took  place.  The  assumptions,  definitions,  design,  delimitations,  and 
limitations  serve  to  clarify  the  terms  of  this  study  and  its  findings.  Chapter  II  reviews  some  of 
the  literature  and  research  addressing  evaluation  of  teacher  performance.  Data  sources,  collection 
techniques,  and  research  procedures  are  outlined  in  Chapter  HI.  Chapter  IV  describes  what  was 
done  during  the  project  and  discusses  the  results.  Chapter  V provides  a summary  of  the  study 
with  implications  for  practice  and  further  research. 
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Chapter  II 


REVIEW  OF  THE  LITERATURE 

Prior  to  submitting  the  pilot  project  proposal,  all  teaching  staff  were  surveyed  to  determine 
perceived  indicators  of  good  teaching.  Immediately  following  this  initial  survey,  central  office 
personnel  reviewed  literature  related  to  good  teaching. 


Teacher  Effectiveness  and  Evaluation 

The  teacher  is  the  key  to  good  schooling.  Being  an  effective  teacher  in  the  times  of  Socrates  and 
Aristotle  was  as  Travers  (1981)  pointed  out  "to  be  a good  person  who  attracted  students"  (p.  14). 
The  prospect  of  relating  good  teaching  to  a single  indicator  in  today’s  educational  context  is 
considerably  more  difficult,  if  not  impossible.  As  King  (1981)  summed  it  up,  "the  more  research 
on  teaching  that  is  done,  the  more  it  is  becoming  evident  that  there  is  no  single  way  to  be  a good 
teacher"  (p.  179).  Questions  concerning  the  indicators  by  which  the  elusive  entity  of  teacher 
effectiveness  are  measured  are  of  central  importance  in  the  realm  of  teacher  evaluation.  Darling- 
Hammond,  Wise,  and  Pease  (1983)  defined  the  evaluation  process  as  "collecting  and  using 
information  to  judge  worth"  (p.  290).  Mitzel  (1960)  underlined  the  significance  of  "teacher 
effectiveness  as  a concept  has  no  meaning  apart  from  the  criterion  measures  or  operational 
definitions  of  success  as  a teacher"  (p.  1482). 

The  analysis  which  follows  is  divided  into  five  major  sections.  The  first  sections  review  some 
of  the  educational  literature  regarding  the  nature  of  teaching  work,  educational  goals  and  models 
of  teaching.  The  objective  is  to  explore  how  these  affect  or  are  affected  by  the  indicators  included 
in  an  evaluation  system.  This  is  followed  by  a discussion  of  the  concept  of  indicators  in  terms  of 
literature  related  to  teaching  effectiveness  and  teacher  evaluation. 
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Influences  on  Teacher  Evaluation  Criteria 


Educational  Goals 

Teacher  assessment  criteria  should  vary  according  to  the  learning  outcomes  considered  most 
appropriate.  According  to  Grimmett  (1982),  the  normative  end  of  the  teaching  spectrum  is 
characterized  "as  consisting  of  a definable  repertoire  of  knowledge,  skills,  and  attitudes  ...  that 
research  has  demonstrated  are  effective  in  leading  to  student  gains"  (p.  48).  The  view  from  the 
professional  end  sees  "teaching  more  as  a clinical  process  ...  involving  reflection  and  informed 
choices  about  which  behaviors  and  strategies  are  deemed  most  appropriate  in  specific  teaching- 
learning  situations"  (ibid,  p.  48).  Though  Grimmett  (like  Darling-Hammond,  Wise,  & Pease, 
1983)  made  a strong  case  for  thinking  of  teaching  tasks  along  a continuous  line  of  increasing 
complexity,  he  conceded  that  a polarized  dichotomy  has  been  prevalent  in  the  past:  at  the 
normative  pole  are  those  with  an  academic  skills  mission  and  at  the  other  are  those  who  are 
inclined  towards  a social  and  personal  development  mission. 


Direct  Instruction  - Content-Oriented  Curriculum 

Those  espousing  the  content-oriented  curriculum  and  a concern  for  productive  use  of  time 
associated  with  the  normative  view,  generally  advocate  direct  instruction  methods.  Grimmett 
and  others  (Centra  & Potter,  1980;  Darling-Hammond,  Wise  & Pease,  1983;  Knapp,  1982) 
indicated  that  the  majority  of  teacher  effectiveness  research  studies  have  been  of  the  direct 
instmction  sort.  What  one  makes  of  this  research  is  determined  to  a great  extent  by  the  views 
one  holds  about  the  desired  learning  outcomes  of  schooling.  One’s  assumptions  about  the  degree 
of  impact  that  teachers  have  on  student  learning  also  enter  the  picture.  Those  who  put  their  faith  in 
this  line  of  research  seek  "context-free  generalizations  about  what  leads  to  or  constitutes  effective 
teaching"  (Darling-Hammond,  Wise,  & Pease,  1983,  p.  293).  Grimmett  (1982)  summarized 
some  generalizable  criteria  from  several  research  studies  and  accepted  these  as  being  useful  to 
teachers.  Additional  support  in  the  literature  consulted  was  found  in  the  works  of  Borich  (1979), 
Gage  (1963),  Medley  (1982),  MacKay  and  Osoba  (1978),  and  Mireau  (1985).  In  reporting 
"process-product  correlations"  in  five  major  studies,  Borich  (1979)  focused  on  practical 
implications  and  derived  a list  of  13  competency  statements,  which  were  seen  as  being  useful 
to  teachers  by  both  Grimmett  and  Mireau.  They  saw  similarly  productive  benefits  to  be  derived 
from  the  seven  "teacher  should"  statements  enumerated  by  Gage  (1963).  Grimmett  (1982) 
contended  that  such  correlational  findings  are  also  useful  to  those  who  favor  the  indirect 
approach. 
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Inductive  Teaching  - Person  and  Process^Oriented  Curriculum 
The  correlational  research  evidence  mentioned  above  "provides  the  base  upon  which  teachers  can 
conduct  professional  deliberations"  (Grimmett,  1982,  p.  47).  Less  direct,  more  inductive 
approaches  with  a person  and  process  curriculum  orientation  are  generally  advocated  by  those 
espousing  the  professional  view.  Instead  of  being  rooted  solely  in  process-product  research, 
exploratory  research  using  ethnomethodological,  phenomenological  and  cognitive  information 
processing  methods  are  consulted  to  describe  "classroom  phenomena  with  a view  to  increasing 
understanding  rather  than  producing  generalizable  statements  about  process-product  correlations" 
(Grimmett,  1982,  p.  66).  Teachers  accept  research  findings  tentatively,  "taking  into  account  the 
complexities  and  dynamic  uncertainties  of  teaching-learning  situations"  (ibid,  p.  67).  Centra  and 
Potter  (1980),  ICnapp  (1982),  and  Darling-Hannuiiond,  Wise,  and  Pease  (1983)  reflected  a similarly 
complex  view  of  the  teaching  environment,  as  well  as  a parallel  view  of  the  lessons  to  be  derived 
from  research. 

Criteria  appropriate  to  judge  teacher  effectiveness  at  one  end  of  the  direct^direct  continuum  may 
not  be  very  appropriate  at  the  other.  Travers  (1981)  noted  that  advocates  of  direct  instruction 
attempt  "to  break  down  teaching  into  a set  of ...  component  competencies"  (p.  20).  He  concluded, 
"If  this  can  be  done  then  the  evaluation  of  teacher  effectiveness  can  be  reduced  to  the  simple 
process  of  determining  whether  the  teacher  does  or  does  not  manifest  the  component  skills  in  the 
classroom"  (ibid,  p.  20).  In  the  opinion  of  Travers  (1981),  those  employing  more  indirect 
methods  "should  probably  be  evaluated  in  terms  of  different  criteria"  (p.  23). 

Models  For  the  Study  of  Teaching 

To  this  point  teacher  effectiveness  has  not  been  classified,  nor  has  it  been  defined.  Before  taking  a 
definitive  step  in  this  direction,  brief  consideration  was  given  to  four  models  of  teaching  to  assist  in 
this  process.  Early  teacher  effectiveness  research  yielded  little  of  benefit.  It  was  not  until  Mitzel 
developed  his  model  of  teaching  which  categorized  criteria  into  four  groups  to  guide  the  study  of 
the  relationships  between  them  that  major  advances  were  made  (MacKay  and  Osoba,  1978; 

Medley,  1982).  In  addition  to  looking  at  this  prototypical  paradigm,  models  by  Dunkin  and 
Biddle,  Centra  and  Potter,  and  Medley  will  also  be  explored. 

MitzeVs  1957  Model 

This  model  served  as  the  conceptual  framework  for  most  of  the  process-product  studies  conducted 
in  the  1960s  and  1970s.  Medley  (1982)  provided  a concise  description  of  this  model: 

Mitzel  defined  four  newly  recognized  classes  of  variables  related  to  teacher 
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effectiveness.  Using  the  terminology  prevalent  at  the  time,  he  referred  to  three 
of  them  as  presage,  process,  and  product  criteria  of  teacher  effectiveness. 

Presage  criteria  included  existing  teacher  characteristics  and  teacher  competencies, 
process  criteria  corresponded  to  teacher  performance  variables,  and  product  criteria 
corresponded  to  teacher  effectiveness.  What  he  called  "context  variables",  included 
a variety  of  situational  factors  that  affect  pupil  learning  but  are  not  controlled  by  the 
teacher  (p.  1483). 

Dunkin  and  Biddle's  1974  Model 

According  to  Grimmett  (1982),  the  significance  of  Dunkin  and  Biddle’s  (1974)  The  Study  of 
Teaching  is  twofold.  First,  it  is  a comprehensive  "review  of  research  on  teaching  up  to  1974;  its 
further  significance  lies  in  the  thoroughness  and  clarity  with  which  the  authors  organized  the  field 
and  conceptualized  teaching  as  an  area  of  study"  (1982,  p.  31).  Grimmett  (1982)  described  the 
model  thus: 

The  model  presumes  a causative  link  between  four  clusters  of  variables:  presage, 
context,  process,  and  product.  Presage  variables  consist  of  the  formative  experience, 
training,  and  personal  properties  that  a teacher  brings  to  the  classroom.  Context  variables 
pertain  to  those  aspects  of  the  school  community,  physical  classroom  set-up  and  pupil 
formative  experiences  and  properties  that  impinge  on  the  teaching-learning  situation. 

Process  variables  apply  specifically  to  what  goes  on  inside  the  classroom,  notably 
teacher  and  pupil  behavior  and  the  changes  in  behavior  that  occur  in  pupils  as  a 
result  of  interaction  with  the  teacher  focused  on  the  curriculum.  Product  variables 
are  the  immediate  and  long-term  effects  of  such  interaction  that  presumably  contribute 
to  student  development  of  skills,  knowledge,  and  attitudes  (p.  31). 

Centra  and  Potter's  1980  Model 

This  conceptual  framework  depicts  the  relationships  among  many  of  the  variables  which  Centra 
and  Potter  (1980)  believed  contribute  to  variance  in  student  learning.  Each  of  the  seven  types  of 
variables  is  grouped  into  three  broader  classifications.  Student  variables  include  student 
characteristics,  student  behavior,  and  student  learning  outcomes.  There  are  two  sets  of  teacher 
variables:  teacher  characteristics  and  teaching  performance.  Within  school  conditions  and  school 
or  school  district  conditions  are  the  other  sets  of  variables. 

The  only  variables  that  have  a direct  effect  on  student  learning  outcomes,  in  their  view,  are  student 
characteristics  and  student  behavior.  "Learning  is  something  children  do . . . not  something  which 
schools  or  teachers  do  to  them"  (Centra  & Potter,  1980,  p.  287).  Teacher  performance  and  within 


9 


school  conditions  act  directly  on  student  behavior  and  have  the  closest  indirect  effect  on  outcomes. 
The  influences  of  the  more  indirect  variables  are  described  by  Centra  and  Potter  (1980)  as: 

Teacher  characteristics  influence  student  behavior  and  learning  only  as  they  affect 
teaching  performance.  And  school  or  school  district  conditions . . . can  be  expected 
to  influence  students  largely  through  teacher  or  teaching  factors,  or  through  within 
school  conditions  (pp.  275-276). 

Medley's  1982  Model 

Medley  (1982)  updated  the  Mitzel  paradigm.  He  included  ’’five  types  of  variables,  each  of  which 
have  been  used  at  one  time  or  another  as  a criterion  for  evaluating  teachers"  (p.  1899).  This 
conceptual  framework  includes  pre-existing  teacher  characteristics,  teacher  competencies,  teacher 
performance,  pupil  learning  experiences,  and  pupil  learning  outcomes.  Acting  upon  these  are 
four  "on-line"  variables  which  are  not  controlled  by  the  teacher:  teacher  training,  external  context 
(school  characteristics),  internal  context  (class  characteristics),  and  individual  pupil  characteristics. 
These  groupings  break  Mitzel's  four  categories  into  finer  categories.  The  most  significant  departure 
is  his  definition  of  effective  teacher  performance  in  terms  of  "a  larger  unit  called  a teaching  strategy 
or  model"  (Medley,  1982,  p.  1899)  rather  than  in  terms  of  single  behaviors. 

All  the  other  cells  shown  affect  pupil  learning  outcomes  directly  or  indirectly.  The  effectiveness 
of  a teacher  depends,  then,  upon  at  least  eight  variables  that  can  be  distinguished  clearly  from  one 
another. 

Studying  these  four  models  and  their  supporting  literature  reveals  at  least  three  major 
similarities  that  are  valuable  in  classifying  and  thinking  about  effective  teaching  criteria. 

1.  Each  set  of  variables  in  the  other  models  can  be  placed  within  one  of 
Mitzel's  four  broad  categories. 

2.  Causative  relationships  between  the  sets  of  variables  are  assumed,  but  not 
proven  in  each  case. 

3.  The  importance  of  other  variables  is  seen  in  relation  to  their  perceived 
impact  upon  student  outcomes. 

Considering  the  major  differences  among  the  models  is  useful  as  well.  An  increasing  emphasis  on 
contextual  variables  is  evident  as  we  move  closer  to  the  present.  The  more  recent  models  conceive 
of  teaching  performance  as  a multidimensional  process  and  suggest  that  patterns,  rather  than 
individual  behaviors,  are  of  the  greatest  concern.  A picture  of  teaching  as  a more  complex  and 
unpredictable  process  appears  to  be  emerging  - one  that  places  teaching  toward  the  professional 
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end  of  the  continuum  referred  to  earlier.  The  model  that  one  uses  to  conceptualize  teaching  has 
an  important  impact  upon  the  criteria  employed  to  evaluate  it. 

Having  looked  into  some  of  the  concepts  related  to  the  nature  of  teaching,  the  focus  now  shifts  to 
the  second  part  of  Darling-Hammond,  Wise,  and  Pease’s  definition,  from  which  we  departed  at  the 
onset  of  this  review.  We  have  seen  how  one’s  assumptions  about  the  nature  of  teaching  work,  the 
goals  that  one  holds  to  be  central  to  education,  and  the  paradigm  by  which  one  conceptualizes 
teaching  all  contribute  to  one’s  judgement  of  teaching  quality.  The  discussion  now  shifts  to  the 
concept  of  teacher  evaluation  criteria  itself. 


Teacher  Evaluation  Criteria 

In  this  section,  the  concept  of  criteria  is  examined  in  a general  way.  Subsequent  to  a short 
consideration  of  some  definitions  of  criteria,  a scheme  for  categorization  is  suggested.  The 
matter  of  criterion  selection  concludes  the  discussion. 

Defining  Criteria 

A sampling  of  the  educational  literature  dealing  with  effectiveness  and  evaluation  reveals  that  terms 
such  as  variables,  measures,  indicators,  competencies,  and  criteria  are  often  used  interchangeably. 
Nevertheless,  there  appears  to  be  some  agreement  on  a more  precise  definition.  Mitzel  (1960) 
clarified  the  way  in  which  the  term  is  most  often  used  with  reference  to  teacher  effectiveness: 

The  term  criterion  is  commonly  attached  to  any  set  of  observations  that  may  be  used 
as  standards  for  evaluative  purposes.  In  this  sense  a criterion  measure  cannot  be 
merely  any  dependent  variable  which  happens  to  be  at  hand.  Calling  a particular 
measure  a criterion  lends  to  it  connotations  of  worth  and  value.  Criteria  cannot  be 
trivial;  otherwise  evaluations  are  made  against  trivial  standards  (pp.  1481-1482). 

Rath  (1982)  distinguished  between  standards  and  criteria  from  a teacher  evaluation  perspective 
and  provided  an  illustration  to  assist  in  making  this  point: 

A criterion  is  a statement  that  describes  or  designates  a variable  of  interest  or  an 
attribute  of  a person  that  is  to  be  considered.  A standard  denotes  the  amount  of 
the  variable  or  attribute  required  to  receive  a specific  grade.  Thus,  if  height  were 
a criterion  for  becoming  a policeman,  a standard  might  be  60  inches.  If  a candidate 
for  the  police  force  met  the  height  criterion  at  a standard  of  60  inches  or  more,  he 
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would  be  graded  ’’eligible” ...  Ideally,  evaluators  would  like  to  be  able  to  specify 
both  the  criteria  and  the  standards  as  clearly  as  was  done  in  this  example  (p.  614). 

The  relationship  between  standards  and  criteria  was  also  expressed  by  Lawton,  Hickcox, 

Leith  wood,  and  Musella  (1986): 

Criteria . . . refer  to  indicators  that  measure  some  quality  or  behavior,  some 
might  be  quite  complex,  such  as  the  style  of  classroom  management,  while 
others  might  be  sti-aightforward,  such  as  punctuality.  Standards  refer  to  the 
level  of  expectations  regarding  criteria  (p.  4). 

Criteria,  then,  must  be  distinguished  from  standards.  Not  only  must  a criterion  be  an  indicator, 
or  a variable,  it  must  be  a meaningful  indicator.  As  criteria  can  be  quite  specifically  defined,  so  too, 
can  they  be  thought  of  as  broad  categories. 


Categories  of  Criteria 

Each  of  the  four  models  of  teaching  mentioned  above  categorizes  effectiveness  variables  in 
accordance  with  its  own  particular  conceptual  frame.  Differences  in  this  regard  appear  to  be 
fewer  than  similarities.  From  our  perspective,  Mitzel’s  three  non-contextual  descriptors  - presage, 
process,  and  product,  are  useful  criteria  grouping  mechanisms  for  two  main  reasons.  First,  their 
almost  thirty  years  of  use  would  lead  one  toward  the  assumption  that  they  are  relatively  widely 
understood  in  the  education  community.  Second,  the  categorizing  labels  used  in  the  other  three 
paradigms  can  all  be  placed  within  one  of  the  Mitzel  domains.  In  fact,  Dunkin  and  Biddle  used  the 
exact  same  terms.  Centra  and  Potter  used  the  heading  teacher  characteristics  for  presage,  student 
learning  outcomes  for  product,  and  the  two  categories,  teaching  performance  and  student  behavior 
for  process.  As  to  how  the  groupings  from  the  third  model  are  related.  Medley  (1982)  made  this 
quite  clear. 

This  model  derives  directly  from  Mitzel’s;  his  presage  criteria  included  pre-existing 
teacher  characteristics,  teacher  training  variables,  and  teacher  competencies;  his  process 
criteria  included  teacher  performance  and  pupil  learning  experiences;his  product  criteria 
were  based  on  pupil  learning  outcomes  (p.  1899). 

The  position  taken  here,  then,  is  that  virtually  every  criterion  by  which  the  quality  of  teaching  can 
be  judged  fits  into  of  the  three  groups:  presage,  process  and  product  Mitzel  has  given  us  a very 
useful  method  of  considering  the  wide  array  of  variables  related  to  teacher  effectiveness. 
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Presage  Criteria 

In  current  teacher  evaluation  presage  criteria  are  widely  used.  Duhamel,  Cyze,  Lamacraft,  and 
Larocque  (1979)  found  that  these  criteria  were  being  used  in  86  percent  of  the  Ontario  systems 
surveyed  in  their  study  (p.  29).  In  the  past,  they  enjoyed  even  wider  application.  "Adaptability, 
considerateness,  enthusiasm,  good  judgement,  honesty,  and  magnetism  were  identified  by  Charters 
and  Waplies  in  the  1929  Commonwealth  Teacher  Training  Study  as  traits  associated  with  the  best 
teachers"  (Medley,  1982,  p.  1895).  Medley  indicated  that  many  of  the  presage  rating  scales  still 
in  use  today  derive  from  a 1930  system  developed  by  Barr,  who  compiled  a list  of  characteristics 
from  studies  of  this  sort  Many  students  of  teacher  evaluation  look  less  favorably  upon  the 
extensive  use  of  presage  criteria.  Detractors  find  two  main  faults.  One  is  the  claim  "that  there  is 
no  persuasive  evidence  that  there  is  any  relationship  between  the  presage  criteria  mentioned  and 
the  teacher’s  effectiveness"  (Duhamel  et  al.,  1979,  p.  29).  The  other  is  expressed  by  Ryan  and 
Hickcox  (1980):  "The  rating  forms  themselves  include  a good  proportion  of  categories  having  to 
do  with  personality,  dress,  appearance,  which  have  little  relation  to  instructional  improvement" 

(p.  21).  Mitzel  (1960)  put  presage  criteria  into  this  perspective:  "precedent  forces  their  consideration 
as  criteria . . . they  are  pseudo  criteria,  for  their  relevance  depends  upon  an  assumed  or  conjectured 
relationship  on  process  or  product"  (p.  1484). 

Advocates  for  the  inclusion  of  these  criteria  in  teacher  evaluation  systems  see  the  situation  somewhat 
differently.  Sweeney  (1980)  felt  that  it  is  important  to  assess  teachers  on  their  humanistic  qualities 
and  spoke  of  the  teacher  as  "an  institutional  representative  (who)  must  carry  the  flag  for  his  school 
and  profession  in  a manner  which  enhances  the  public  image  and  status  of  both"  (p.  12).  A slightly 
different  statement  along  this  line  of  thought,  was  expressed  by  King  (1981): 

Whether  they  are  used  in  personnel  decisions  or  in  self-evaluation  for  professional 
growth,  indirect  measures  suggest  an  appropriately  multidimensional  model  for 
teacher  evaluation.  Because  they  can  provide  valuable  information  unavailable  ft-om 
more  direct  sources,  indirect  measures  may  prove  invaluable  (p.  179). 


Process  Criteria 

An  overwhelming  mass  of  research  on  teaching  has  sought  to  identify  teacher  and  student  behaviors 
that  influence  learning.  "It  is  not  surprising,  therefore,  that  process  indicators  are  the  most  frequently 
used  criteria  in  present  evaluation  practices"  (Duhamel  et.  al.,  1979,  p.  28).  The  significant 
assumption  upon  which  the  use  of  these  criteria  is  based  is  "that  congruence  with  these  behavioral 
norms  produces  effective  teaching"  (ibid,  p.  27).  Many  of  the  consulted  scholars  feel  that  several 
behaviors  have  shown  a consistently  high  correlation  with  learning  outcomes  and  should. 
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therefore,  be  included  as  criteria  to  determine  teacher  quality.  Still  others  believe  that  patterns  of 
teacher  behavior  may  be  more  significant  (Centra  and  Potter,  1980;  Medley,  1982).  Efforts 
continue  to  refine  observation  techniques,  establish  validity  and  reliability,  and  make  the  process 
yet  more  objective.  Evertson  and  Holley  (1981)  thus  described  the  status  of  classroom 
observation: 

Classroom  research  has  recently  made  progress  in  identifying  process  variables 
that  are  good  indicators  of  effective  teaching  as  measured  by  achievement.  A picture 
of  what  effective  teaching  looks  like  in  the  classroom  is  beginning  to  emerge,  and  the 
assessment  of  teachers  by  means  of  observation  can  now  be  regarded  as  a meaningful 
activity  (p.  107). 

Apprehensions  over  the  teacher  evaluation  applications  of  performance  criteria  generally  have 
to  do  with  their  overuse  or  misuse.  Hickcox  (1982),  among  others,  cautioned  against  exclusive 
reliance  on  observations:  ”We  know  for  certain  that  a principal  sitting  in  the  back  of  the  room 
with  a notebook  and  a rating  sheet  once  a year  or  once  every  three  years  is  not  going  to  produce 
a rating  that  has  any  reliability  or  validity  (p.  7).  "Observer  bias,  insufficient  sampling  of 
performance,  and  poor  measurement  instruments  can  threaten  the  reliability  and  validity  of 
results, " warned  Darling-Hammond,  Wise,  and  Pease  (1983,  p.  306). 

Some  of  those  viewing  teaching  at  the  professional  end  of  the  spectmm  have  additional 
reservations.  Rath  (1982)  questioned  the  usefulness  of  "low  inference"  data.  Eisner  (1977) 
used  this  example  to  illustrate  the  limitations  of  relying  too  heavily  upon  objective  data: 

A behavioral  description  of  an  eyelid  closing  on  the  left  eye  at  a rate  of  two 
closures  per  second  could  be  described  in  just  that  way.  But  a thick  description 
of  such  behaviors  within  the  context  of  a cultural  subsystem  could  be  described 
as  a wink  (p.  353). 

This  example  is  perhaps  overstating  the  case.  Others  who  perceive  teaching  in  a similar 
professional  sense  would  see  possibilities  for  growth  in  the  use  of  low-inference  information. 
Teachers  would  draw  their  own  conclusion  from  such  data,  in  this  view. 
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Product  Criteria 

In  contrast  to  the  other  two  groupings,  the  use  of  product  variables  is  not  very  widespread. 
Varieties  of  this  approach  assume  that  at  least  a portion  of  student  learning  can  reasonably  be 
attributed  to  the  efforts  of  teachers.  Haefele  (1980)  and  Darling-Hammond,  Wise,  and  Pease 
(1983)  identified  a number  of  results-oriented  approaches.  Among  these  methods  are  three 
variations  which  assess  teacher  effectiveness  on  the  basis  of  standardized  tests.  On  such 
approaches,  the  latter  article  commented,  "Studies  of  the  reliability  of  student  test  scores  as  a 
measure  of  teaching  effectiveness  consistently  indicate  that  reliability  is  quite  low,  that  is,  that  the 
same  teacher  produces  markedly  different  results  in  different  situations"  (Darling-Hammond,  et  al., 
1983,  p.  307).  The  final  outcome  method  discussed  and  advocated  by  Haefele  is  a negotiated 
approach,  "The  teacher,  together  with  the  principal  and/or  curriculum  supervisor,  establishes 
mutually  agreed  upon  (negotiated)  instmctional  goals  for  the  year"  (1980,  p.  352). 

The  difficulty  in  linking  student  outcomes  to  teaching  performance  was  also  recognized  by  Centra 
and  Potter  (1980),  who  noted  that  "student  achievement  is  affected  by  a considerable  number  of 
variables,  of  which  teacher  behavior  is  but  one"  (pp.  286-287).  Several  studies  indicate  a general 
teacher  opposition  to  the  inclusion  of  these  criteria  in  evaluation  processes  (Knapp,  1982;  Lawton 
et  al.,  1986;  Ryan  & Hickcox,  1980). 

Very  intentionally  there  has  been  an  effort  to  steer  away  from  an  encyclopedic  listing  of  the  myriad 
of  criteria  which  could,  in  some  way,  be  utilized  to  assess  one  aspect  of  teacher  quality  or  another. 
Rather,  the  decision  is  to  touch  upon  the  three  broad  categories  in  a more  general  manner.  Limiting 
or  selecting  the  appropriate  number  of  criteria  which  can  legitimately  be  applied  to  the  assessment 
of  teacher  quality  in  any  given  situation  is  a matter  addressed  next 

Selecting  Teacher  Evaluation  Criteria 

Judicious  selection  of  criteria,  in  our  view,  should  be  undertaken  only  after  the  following  questions 
have  been  carefully  considered.  What  are  the  purposes  of  evaluation?  Who  decides  which  criteria 
will  be  used?  Do  the  criteria  apply  to  all  teachers  or  is  their  appropriateness  based  more  specifically 
on  the  unique  nature  of  individual  teaching  assignments?  Answers  to  the  final  two  questions  are 
dependent,  to  a great  extent,  upon  the  stated  purposes  of  assessment. 
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Duncan  (1984)  listed  the  eight  purposes  for  teacher  evaluation  delineated  in  Alberta  Education's 
Program  Policy  Manual: 

1 . To  promote,  achieve,  and  maintain  an  acceptable  quality  of  instmction. 

2.  To  help  improve  performance  of  teachers  in  securing  desirable  instruction  outcomes. 

3 . To  provide  information  which  will  be  useful  when  considering  placement  of  staff, 
transfer,  retention,  promotion,  tenure,  and  permanent  certification. 

4.  To  provide  specific  feedback  concerning  teacher  performance. 

5 . To  provide  professional  assistance  to  teachers  in  the  performance  of  teaching  tasks. 

6.  To  provide  a basis  for  planning  inservice  programs. 

7 . To  provide  written  evaluation  reports  on  teachers  in  public  and  private  schools  for 
purposes  of  documentation,  as  required. 

8 . To  assist  teachers  in  professional  growth  and  development  (pp.  4-5). 

Teacher  evaluation  procedures  are  generally  divided  into  two  categories  on  the  basis  of  purpose. 
The  two  broad  purposes  are  generally  labelled  either  summative  (for  administrative  decisions)  or 
formative  (for  improvement  of  teaching).  The  first,  third  and  seventh  of  the  Alberta  provincial 
level  purposes  would  appear  to  fit  into  the  summative  category,  while  the  remainder  would  appear 
to  fit  better  into  the  formative  category. 


Formative  Evaluation 

Keeler  contended  "The  teacher  must  play  a lead  role  - or  at  least  a co-stairing  one  - in  formative 
evaluation,"  (1980,  p.  30).  He  thought  that  evaluation  "is  on-going,  should  include  self- 
evaluation,  and  should  supply  reports  only  to  the  teacher  involved"  (ibid,  p.  30). 

Townsend  (1984)  believed  effective  models  of  formative  evaluation  have  six  fundamental 
characteristics.  They  are: 

1 . Growth  oriented  as  opposed  to  deficiency  based. 

2 . Collegially  developed  and  implemented  as  opposed  to  a unilateral,  administratively 
developed  program. 

3 . Data  based  as  opposed  to  myth  bound. 

4.  Peer  group  and  total  faculty  appraisal  as  opposed  to  the  unilateral  assessment  of  a 
teacher  by  an  individual  evaluator. 
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5 . A holistic  and  complex  view  of  teachers  and  teaching  as  opposed  to  an  atomistic  and 
simplistic  perspective  that  focuses  on  teachers'  personal  attributes,  classroom 
appearance,  or  lessons  plans. 

6.  A situational  as  opposed  to  a universal  perspective  (p.  27). 


Summative  Evaluation 

It  can  be  argued  that  a summative  evaluation  of  teacher  performance  must  be  taken  prior  to 
embarking  upon  any  meaningful  program  of  improvement.  Scriven  (1981)  stated  this  case: 

Summative  evaluation  is  primary  because: 

(1)  human  careers  are  at  stake,  not  "mere"  improvement; 

(2)  if  it  is  not  possible  to  tell  when  teaching  is  bad  (or  good)  overall,  it  is  not  possible  to 
tell  when  it  has  improved; 

(3)  if  it  is  possible  to  tell  when  it  is  bad  or  good,  personnel  decisions  can  be  made  even 
though  it  is  not  known  how  to  make  improvements. 

In  short,  diagnosis  is  sometimes  easier  than  healing,  and  an  essential  preliminary 

to  it  (p.  244). 

The  importance  of  criteria  and  standards  appear  to  be  central  to  the  legal  context  of  summative 
evaluation.  Beckham  (1982)  wrote  of  the  requirements  of  this  status-related  form  of  evaluation, 
listing  fifteen  guidelines  to  be  followed.  Five  of  these  relate  to  criteria: 

1 . Written  policies  defining  evaluation  criteria  that  are  reasonable,  related  to  the 
teacher's  job  performance  and  the  school  system's  goals  and  objectives. 

2 . Criteria  for  evaluating  teacher  performance  should  be  developed  from  a 
thorough  job  analysis  and  shown  to  be  job-related  through  content  or  empirical 
validation  procedures. 

3 . Where  possible,  it  is  preferable  that  evaluation  criteria  be  objective  or,  where 
not  obviously  objective,  susceptible  to  objective  assessment 

4.  Subjective  criteria  ratings  should  be  considered  as  but  one  component  in  an 
overall  evaluation  system. 

5 . Teachers  should  be  fully  informed  of  the  criteria  (pp.  11-12). 

Duncan  (1984)  derived  a similar  list  by  soliciting  the  opinions  of  five  identified  experts  in  the  field. 
The  single  criteria-related  item  on  his  list,  which  is  not  also  found  above,  is  the  requirement  that  a 
"variety  of  objective  data  must  be  collected"  (p.  79). 
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Assessment  of  Teacher  Performance 


Effective  schools  research  has  found  strong  correlations  among  principal  expectations  for  teachers, 
teacher  expectations  of  students,  and  student  achievement  In  effective  schools  the  role  of  the 
principal  is  that  of  instmctional  leader  rather  than  manager.  As  such,  his/her  role  is  to  establish 
with  staff  specific  learning  expectations,  deliver  to  teachers  the  necessary  materials  to  carry  out  the 
instmctional  programs,  and  to  continuously  evaluate  the  level  of  mastery  of  students  and  staff 
(Brookover  et  al,  1982;  Brookover  & Lezotte,  1979;  Leithwood  & Montgomery,  1982; 

Manasse,  1984). 

Despite  the  importance  of  teacher  appraisal,  scholars  and  practitioners  perceive  it  as  ineffective 
in  improving  the  quality  of  classroom  instmction.  One  of  the  principal  weaknesses  identified  in 
research  is  the  inability  to  connect  specific  teacher  behaviors  with  student  outcomes  (Acheson  & 
Gall,  1987;  Bartalo,  1988,  Calabrese,  1986).  A second  weakness  is  the  inability  of  the  principal 
to  distinguish  between  effective  and  ineffective  instmction  (DeRoche,  1981;  Johnson  & Snyder, 
1986;  Leithwood  & Montgomery,  1982;  O’Neill  & Shoemaker,  1989).  A third  problem  is  the 
lack  of  consistency  among  appraisers  as  to  what  is  effective.  When  variances  in  the  interpretation 
of  a teaching  segment  were  noted,  effective  teaching  seemed  to  be  based  on  principal  taste  more  so 
than  on  sound  pedagogy  (Calabrese,  1986;  Furst,  1971;  Soar,  Medley  & Coker,  1983)  A fourth 
difficulty  is  the  persistence  of  an  adversarial  relationship  between  the  teacher  and  principal  in  the 
evaluation  of  classroom  performance.  Teacher  appraisal  was  frequently  perceived  as  a weapon  for 
fault  finding  rather  than  a group  process  for  problem-solving  (Acheson  & Gall,  1981;  Bartalo, 
1988;  O’Neill  & Shoemaker,  1989;  Soaretal.,  1983). 

Numerous  researchers  have  attributed  the  weaknesses  to  the  lack  of  appropriate  training.  A 
program  that  involves  principals  and  lead  teachers  in  a collaborative  effort  to  assess  instmctional 
programs  and  to  recommend  reform  would  address  the  concerns  expressed  above.  Such  a 
program  requires  adequate  time  to  be  understood,  provision  for  the  application  of  skills,  and  the 
establishment  of  effective  collegial  relationships  (Hunter,  1988;  Klitgaard,  1987;  Manatt,  1988). 


Other  Factors  Related  to  Criteria 

A number  of  additional  matters  and  considerations  enter  into  the  discussion.  Though  a number 
of  voices  call  for  our  jurisdiction  to  establish  one  set  of  criteria,  the  wisdom  of  so  doing  is  doubted 
by  many  others,  as  expressed  by  Hickcox  (1982):  ’’The  difference  between  teaching  kindergarten 
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in  an  inner  city  school  and  high  school  physics  in  a affluent  suburb  must  be  considerable"  (p.  9). 
Knapp  (1982)  and  Darling-Hammond,  Wise,  & Pease  (1983)  felt  that  using  the  same  criteria  for 
both  formative  and  summative  purposes  is  a problem.  Knapp  (1982)  stated  "Those  who  write 
about  resistance  to  evaluation ...  (argue)  that  summative  evaluation  of  teachers  tends  to  work 
against  the  efforts  to  alter  their  behavior  by  generating  insecurity  and  defensive  behavior"  (p.  7). 
Common  sense  and  a knowledge  of  organizational  theory  would  strongly  suggest  that  teachers 
should  play  a large  part  in  helping  to  set  the  criteria  by  which  their  work  is  to  be  judged.  Clear 
communication  of  exactly  what  is  meant  by  each  criterion  used  to  judge  the  quality  of  teachers  is 
another  factor  of  the  utmost  importance  to  the  eventual  success  of  the  whole  process.  The  Spirit 
River  Educational  (Quality  Indicators  Project  involved  teachers  in  establishing  criteria  by  which 
they  would  be  evaluated  and  clearly  communicated  these  criteria  to  them. 
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Chapter  III 


METHODOLOGY 

This  chapter  describes  the  specifics  of  the  methodology  employed  in  the  project  It  provides  a 
detailed  account  of  what  was  done  during  the  three  years  of  the  project. 


Year  One:  Development 

A proposal  of  intent  to  participate  in  a pilot  project  in  Alberta's  Educational  Quality  Indicators 
(EQI)  initiative  was  submitted  to  Alberta  Education  in  December  1988.  During  the  15  months  that 
followed,  administrative  staff  spent  many  hours  in  discussion  and  debate  attempting  to  refine  and 
focus  the  project  within  the  terms  of  reference  and  manageable  limits  for  a small  jurisdiction.  The 
proposal  focused  on  the  development  of  criteria  for  teacher  effectiveness  in  a formative  process. 

Experience  and  an  initial  literature  review  strongly  indicated  that  for  any  project  involving  our  key 
components  of  action  research,  development  of  criteria  and  formative  evaluation,  it  was  essential 
to  gain  the  acceptance,  involvement,  and  positive  initiative  of  staff.  In  a demonstration  of  good 
faith,  the  Board  supported  the  concept  by  adding  a staffing  component  to  all  small  schools  in  the 
1989-90  budget  and  making  the  EQI  project  a priority  in  the  Division's  1989-90  Three  Year  Plan. 
Early  in  June  1989,  the  superintendent  sent  a memo  to  all  teaching  staff  outlining  the  proposed 
project,  its  intent,  and  asking  for  volunteers  to  take  a leadership  role.  A survey  a week  later  asked 
teachers  to  indicate  the  criteria  each  felt  were  critical  to  improving  teacher  effectiveness.  The 
response  from  the  staff  was  positive  and  encouraging.  The  teacher  survey  on  criteria  received 
over  an  80%  response.  Eight  experienced  and  respected  teachers  formed  a local  committee. 

During  the  summer  of  1989,  central  office  administrators  conducted  a thorough  search  of  the 
literature  relevant  to  the  components  of  the  project.  Key  articles  on  action  research,  formative 
evaluation,  and  quality  indicators  were  selected  and  distributed  to  the  local  committee.  Responses 
to  the  June  teacher  survey  on  criteria  were  collated. 
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Table  1 


Indicators  of  Quality  Instruction 


1.  Sequentially  Developed  Planning  and  Preparation 

1 . 1 The  teacher  maintains/develops  short,  unit  and  long  term,  flexible  plans  with  an  evaluation 
process. 

1 .2  The  teacher  adheres  to  Curriculum/Program  of  Studies  documents. 

1 .3  The  teacher  provides  a safe  and  organized  environment  with  resources  and  equipment  readily 
available. 

1 .4  The  teacher  provides  for  individual  learning  needs  with  consideration  of  student  abilities, 
based  on  diagnostic  information. 

1 .5  The  teacher  provides  a variety  of  learning  activities. 

1 .6  The  teacher  ensures  that  objectives  are  stated. 

1 .7  The  teacher  provides  for  motivation  of  students. 

1 .8  The  teacher  makes  learning  activities  relevant 

1 .9  The  teacher  plans  for  interdisciplinary  activities. 


2.  Key  Instructional  Strategies 

2. 1 The  teacher  circulates  during  pupil  activities. 

2.2  The  teacher  presents  clear  assignments  which  are  relevant  to  student  experiences. 

2.3  The  teacher  exhibits  classroom  management  skills. 

2.4  The  teacher  provides  an  environment  whereby  active  learning  takes  place  within  a working 
and  non-threatening  atmosphere. 

2.5  The  teacher  provides  a physical  classroom  environment  which  is  stimulating,  enriched, 
relevant  and  has  displays  of  student  work. 

2.6  The  teacher  exhibits  a variety  of  questioning  techniques. 


3 .  Communication  Skills 

3 . 1 The  teacher  provides  clear  explanation  of  objectives,  assignments  and  behavioral  expectations. 

3 .2  The  teacher  exhibits  a variety  of  positive  reinforcement  techniques. 

3 .3  The  teacher  is  an  active  hstener. 

3 .4  The  teacher  provides  timely  and  relevant  reviews. 

3 .5  The  teacher  uses  language  appropriate  to  the  developmental  level  of  students. 

3 .6  The  teacher  maintains  regular  parental  information  flow. 


4 .  Pupil/Teacher  Relationships 

4 . 1 Mutual  respect  is  exhibited  between  teacher  and  students. 

4.2  Clear  routines,  expectations  and  rules  are  made  known  and  followed. 

4.3  The  teacher  provides  for  positive  peer  interaction  and  support. 

4.4  The  teacher  uses  a fair  and  meaningful  evaluation  process. 

4.5  The  teacher  provides  prompt  feedback  on  student  assignments. 
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In  August  1989,  the  local  committee  and  the  superintendent  met  with  the  purpose  of  finalizing  the 
criteria  and  establishing  the  process  and  terms  of  reference  for  the  project  within  the  established 
guidelines.  This  local  committee  reviewed  pertinent  research  and  literature,  developed  grouped 
sets  of  quality  indicators  of  effective  teaching  (See  Table  1),  formulated  key  variables  which  could 
impact  the  effectiveness  of  implementation,  articulated  clear  explanations  of  each  indicator  and  gave 
suggestions  on  data  gathering  for  each.  They  reached  agreement  on  desired  outcomes,  developed 
data  gathering  sheets  for  indicators,  variables,  and  outcomes,  and  held  a mini- workshop  on 
formative  evaluation.  They  then  finalized  a draft  document  for  presentation  to  all  teaching  staff, 
and  developed  an  action  plan  and  consequent  logistics  and  timelines  for  implementation  in  all 
division  schools  on  a voluntary  basis. 

The  draft  project  documents  were  presented  to  the  Administrative  Council  at  its  initial  meeting  in 
August  1989,  and  local  committee  members  introduced  the  project  to  teaching  staffs  at  the  staff 
meeting  in  August  1989.  Staff  and  school  administrators  were  requested  to  give  input  on  the  draft 
documents  to  local  committee  members  by  September  1989. 

The  local  committee  met  to  review  the  input  from  staff  and  subsequently  finalize  the  working 
project  documents  for  use  in  the  first  year  of  the  project.  (See  Appendix  A). 

Local  committee  members  presented  a half-day  workshop  to  members  of  the  Administrative 
Council  and  presented  the  keynote  session  to  all  staff  during  the  Divisional  Professional 
Development  Day  in  October  1989.  Formal  presentations  of  the  project  documents  were  made 
by  local  committee  members  to  all  school  staffs  during  the  professional  development  component 
of  the  October  regular  staff  meetings. 

The  finalized  project  for  Year  One  was  implemented  in  all  schools  on  a voluntary  basis  effective 
October  11th,  1989. 

The  local  committee  of  eight  teachers  and  the  superintendent  met  in  late  January  1990  to  review, 
analyze,  and  collate  the  information  received  from  staff  who  had  been  participating  in  the  project 
for  the  first  four  months.  During  this  initial  period,  all  staff  had  the  opportunity  to  meet  voluntarily 
and  discuss  the  previously  developed  indicators  of  effective  teaching  and/or  to  visit  each  other’s 
classrooms  to  observe  one  or  more  of  the  specific  criteria.  Data  were  collected  on  staff 
participation,  specific  indicators  that  were  discussed  and  observed,  local  or  out-of-school  contacts. 
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and  years  of  experience  of  participants.  Information  was  also  gathered  on  geographic  distance, 
use  of  extra  time  provided  to  each  school,  staff  attitudes,  and  extended  professional  development 
activities. 

To  gain  parents'  and  students'  perspectives  of  indicators  of  quality  teaching,  two  survey 
instruments  were  developed  (see  Appendix  B).  In  the  spring  of  1990,  these  surveys  were 
sent  to  all  division  students  in  grades  9, 10, 1 1 and  12,  and  to  parents  of  students  in  grades 
2, 5,  8 and  11.  In  the  two  subsequent  years  the  student  surveys  were  completed  in  scheduled 
classroom  time.  Parent  siuveys  were  sent  to  homes  via  students  in  school.  A teacher  survey  on 
satisfaction  and  professional  development  involvement  was  also  conducted  each  year  (Appendix 
C).  The  data  gathered  at  the  conclusion  of  each  year  were  collated  and  presented  at  the  divisional 
level,  thereby  assuring  the  anonymity  of  respondents. 

At  the  first  annual  meeting  of  the  local  committee,  consensus  was  reached  on  a number  of 
decisions  which  would  become  an  integral  part  of  the  process  for  the  second  year  of  the  project: 

1 . That  the  criteria  remain  the  same  during  year  one. 

2.  That  board  approval  be  sought  to  change  the  Evaluation  Policy  to  allow  the  summative 
evaluation  process  to  use  the  same  criteria  as  the  formative  process,  with  the  addition  of 
the  personal  qualities  as  highlighted  in  the  teacher  handbook  distributed  at  the  introduction 
of  the  project. 

3 . That  extra  funding  be  provided  by  the  division  to  be  used  for  interschool  EQI  visitations 
with  priority  to  specialist  teachers  and  teachers  with  split  grades. 

4.  That  each  school  designate  an  EQI  coordinator  who  will  become  part  of  the  expanded 
local  committee.  These  individuals  will  be  given  designated  EQI  time  to  gather  data  on 
staff  usage. 

5 . That  principals  review  methods  of  allocating  EQI  time  to  ensure  maximum  opportunity 
for  use  by  all  teachers. 

6 . That  each  principal  make  some  component  of  the  EQI  project  one  of  his/her  school’s 
objectives  for  1990-91. 

7 . That  a session  on  formative  evaluation  concepts  and  process  be  offered  at  the  local 
teachers’  professional  development  day  in  October  1990. 

It  was  felt  that  the  above  recommendations  would  ensure  the  continuing  importance  of  the  EQI  project 
to  the  division. 
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Year  Two:  Implementation 


All  seven  recommendations  for  Year  Two  were  acted  upon  early  in  the  1990-91  school  year. 

With  a change  in  superintendent  in  December  1990,  the  goals  and  intent  of  the  project  remained 
essentially  the  same,  with  the  one  significant  change  being  an  expansion  of  the  data  collection 
process. 

A decision  was  made  to  enhance  the  credibility  of  our  project  as  a research  endeavor  through 
the  establishment  of  a teacher  performance  baseline  to  which  comparisons  could  be  made.  This 
represented  a significant  shift  in  focus,  from  voluntary  participation  to  compulsory  participation 
in  this  one  aspect  of  the  project  The  establishment  of  a teacher  performance  baseline  required  an 
assessment  of  teachers'  current  performance.  To  increase  the  consistency  of  teacher  observations 
and  to  define  different  levels  of  teacher  effectiveness,  a "teacher  observer"  from  each  school 
participated  in  a three-hour  training  session  to  establish  indicator  standards.  Using  the  tapes 
produced  by  Laurie  Mireau  (1985)  in  her  doctoral  program  on  teacher  effectiveness,  the  teacher 
performances  observed  were  rated  on  a five-point  scale.  An  attempt  was  made  to  reach  consensus 
about  the  meaning  of  each  of  the  five  scale  points.  The  sample  tapes  then  served  as  a set  of 
indicator  standards  against  which  teacher  performance  was  judged.  During  an  assigned  one- week 
period,  "teacher  observers"  were  released  from  their  instructional/administrative  duties  to  visit 
classrooms  for  a 40  minute  period  to  rate  teacher  performance.  The  "Teacher  Performance 
Tally  Sheet"  (Appendix  D)  was  used  to  record  their  observations. 

All  teachers  and  administrators  involved  in  the  observation  of  teacher  and  student  behavior  were 
given  inservices  on  what  to  look  for  when  observing,  and  how  to  record  the  observation. 

Lectures  as  well  as  videotapes  were  used  to  show  a variety  of  situations  and  strategies  to  be 
examined.  Group  discussion  and  strategies  were  put  into  place  to  do  the  observations.  This 
information  was  taken  back  to  the  schools  and  shared  with  the  staffs  in  order  to  validate  the 
procedure. 

A need  was  also  recognized  for  additional  student  outcome  measures  relevant  to  the  intent  of  the 
indicator  system.  Consideration  was  given  to  the  development  of  our  own  instrument  but  we 
agreed  we  lacked  the  resources  and  expertise  to  tackle  such  a complex  activity.  The  alternative  was 
to  consult  the  Educational  Quality  Indicators  Inventory  of  Assessment  Instruments  (Alberta 
Education,  1990).  Sample  copies  of  a number  of  assessment  instruments  were  accessed.  The 
local  committee  established  guidelines  for  the  selection  of  suitable  instruments: 
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1 . Will  the  instrument  give  us  student  outcome  measures  relevant  to  the  intent  of  the 
indicator  systems? 

2 . Can  the  instrument  be  administered  with  relative  ease? 

3 . Are  norms  provided  to  which  comparisons  can  be  made? 

The  Classroom  Environment  Scale  (Moos  & Trickett,  1987)  was  selected  for  grades  5,  8 and  1 1 
and  the  Student's  Perception  of  Ability  Scale  (Boersma  & Chapman,  1977)  was  selected  for 
grade  2 as  the  instruments  most  closely  meeting  the  criteria  set  (Appendices  E and  F).  Both 
instmments  measured  student  outcomes  in  the  affective  domain,  supplementing  data  gathered 
on  cognitive  outcomes. 

Table  2 summarizes  the  data  collected  during  the  last  two  years  of  the  project. 


Year  Three:  Refinement 

At  the  second  annual  meeting  of  the  local  committee,  the  following  recommendations  were  made 
for  the  final  year  of  the  project: 

1.  That  the  criteria  remain  the  same  as  the  previous  year,  with  the  exception  of 
some  minor  changes  in  wording. 

2.  That  the  division's  summative  evaluation  process  continue  to  use  the  same 
criteria  as  EQI's  formative  criteria. 

3.  That  extra  funding  continue  to  be  provided  by  the  division  to  be  used  for 
out-of-school  EQI  visitations  subject  to  the  principal’s  and  superintendent's 
approval. 

4.  That  each  school  designate  an  EQI  coordinator  who  will  be  responsible  for  data 
collection  and  EQI  promotion. 

5.  That  the  school  administrator  and  EQI  coordinator  provide  to  the  deputy 
superintendent  a plan  for  the  utilization  of  the  four  periods  a week  of  EQI  time. 

6.  That  the  EQI  project  be  one  of  the  school’s  objectives  for  1991-92  (May  be 
one  or  more  of  the  seven  identified  indicators  from  the  Teacher  Performance 
Baseline.) 

7.  That  the  Steering  Committee  continue  to  refine  the  teacher  effectiveness 
indicators  as  required. 
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Table  2 

Data  Collection  1990  to  1992 


Type 

Indicators 

Measures 

Sources 

Staff 

participation 

number  of  visitations 
made  to  other  teachers’ 
classrooms 

teachers 

professional 

development 

number  of  teachers 
attending  out-of-division 
PD  activities 

teachers 

teacher 

effectiveness 

aggregate  rating  on  each  of 
the  26  teaching  behaviors 

evaluator 

Parents 

perceptions 

survey  of  parents  of 
students  in  grades 
2,  5,  8,  11 

parents 

Students 

perceptions 

survey  of  students  in 
grades  9,  10,  11,  12 

students 

attitudes 

Student's  Perception  of 
Ability  Scale  (grade  2) 
Classroom  Environment  Scale 
(grades  5,  8,  11) 

students 

achievement 

grades  3,  6 and  9 
achievement  tests 
grade  12  diploma 
examinations 

students 

discipline 

suspensions,  expulsions, 
referrals  to  office,  dropouts 

administrators 
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8.  That  we  address  the  seven  identified  indicators  in  the  Teacher  Performance 
Baseline  through  a variety  of  professional  development  activities.  (1.6, 1.9, 

2.1,  2.5,  2.6,  3.2,  3.4) 

9.  That  the  data  collected  during  Year  Three  be  the  same  as  that  of  Year  Two. 

10.  That  a designated  contribution  be  made  available  to  the  local  Professional 

Development  Committee  to  support  EQI  related  workshops/speaker  during 
the  fall  Professional  Development  Day  and  other  appropriate  times. 

The  focus  of  the  Spirit  River  Educational  Quality  Indicators  Project  was  on  teaching  behaviors 
and  their  impact  on  student  outcomes.  Through  this  project  a myriad  of  professional  growth 
opportunities  not  normally  available  have  been  provided.  This,  in  turn,  has  resulted  in  the  potential 
for  modification  of  teaching  behaviors. 

We  believed  that  modification  of  specific  teaching  behaviors  would  have  a positive  impact  upon 
desirable  student  outcomes.  It  was,  therefore,  deemed  appropriate  to  compare  data  gathered  on  the 
various  variables  to  determine  the  extent  to  which  there  appears  to  have  been  a conscientious  effort 
to  modify  teaching  behaviors  and  the  subsequent  impact  this  has  had  on  student  outcomes. 
Conclusions  in  the  form  of  findings  were  then  derived  from  the  comparisons  of  the  data  collected. 
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Chapter  IV 


FINDINGS 

This  chapter  contains  four  major  sections  - teacher  participation,  teacher  performance,  perceived 
importance  of  the  criteria  (parent  and  student  perceptions)  and  student  outcomes. 


Teacher  Participation 

Critical  to  the  success  of  this  project  was  the  acceptance,  involvement  and  positive  initiative  of  the 
teaching  staff.  Table  3 is  a list  of  teaching  experience  of  the  teachers  in  the  division.  A record  of 
EQI  visits  was  maintained  at  each  school  and  submitted  to  the  EQI  coordinator  biannually  (see 
Appendix  A,  "Educational  Quality  Indicators  Record  Sheet").  Table  4 presents  the  number  and 
percent  distribution  of  teacher  visitations  during  each  of  the  three  years.  The  number  of  classroom 
visitations  made  each  year  is  indicative  of  the  support  the  project  enjoyed. 

The  concept  of  collegial  classroom  visitations  may  have  been  approached  with  some  caution 
initially,  but  was  enthusiastically  embraced  during  the  second  year  when  visitations  more  than 
tripled,  waning  slightly  in  the  third  year.  The  board's  commitment  to  the  EQI  project,  evident  in 
the  funding  it  provided  to  cover  the  cost  of  substitute  teachers,  the  addition  of  a staffing  component 
to  all  small  schools,  and  the  priority  given  the  project  in  the  district’s  Three  Year  Plan  all 
contributed  to  the  high  level  of  voluntary  participation  in  the  last  two  years. 

In  the  first  two  years,  EQI  contacts  suggested  a direct  correlation  to  years  of  experience,  with  the 
highest  participation  rate  being  that  of  our  most  and  least  experienced  teachers.  This  suggested 
that  visitations  may,  in  fact,  have  been  between  these  two  groups.  During  all  three  years  it  was 
the  most  experienced  teachers  who  made  most  extensive  use  of  the  provision  made  for  visitations 
with  colleagues(see  Table  4).  This  was  to  be  expected  as  it  is  this  group  of  teachers  that  constitute 
approximately  half  of  the  total  teaching  staff  (see  Table  3). 
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Table  3 


Composition  of  Teaching  Staff  In  Years  of  Experience  (Percent) 


Years  of 
Experience 

1989-90 

1990-91 

1991-92 

0-2 

22 

20 

14 

3-5 

8 

14 

18 

6-8 

19 

12 

9 

9-10 

5 

10 

7 

11+ 

46 

45 

53 

Total 

100 

101 

101 

Table  4 

Number  and  Percent  of  Teacher  Classroom 

Visitations 

by  Years  of  Teaching  Experience^ 

Years  of 

1989-90  School  Yr. 

1990-91 

School  Yr. 

1991-92  School  Yr. 

Experience 

Visitations  Percent 

Visitations  Percent 

Visitations  Percent 

0-2 

43 

19.7 

82 

15.1 

56  9.1 

3-5 

12 

5.5 

31 

5.7 

131  21.2 

6-8 

41 

18.8 

87 

16.1 

30  4.9 

9-10 

23 

10.6 

80 

14.8 

31  5.0 

11+ 

99 

45.4 

262 

48.3 

371  59.9 

Total 

218 

100.0 

542 

100.0 

619  100.1 

1 There  were  82  teachers  in  1989-90, 87  teachers  in  1990-91  and  86  teachers  in  1991-92. 
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The  participation  rate  of  teachers  with  0-2  years  of  experience  dropped  substantially  in  the  last 
year.  This  may  be  partially  accounted  for  by  the  fact  that  this  group  comprised  a significantly 
smaller  proportion  of  the  total  teaching  staff  than  it  formerly  did.  Whereas  22  percent  of  teachers 
were  in  the  0-2  year  experience  range  in  1989-90,  this  declined  to  14  percent  in  1991-92. 

The  group  with  3 to  5 years  of  experience  is  the  second  largest  comprising  18  percent  of  the  total 
teacher  population.  It  is  this  group  that  also  accounted  for  the  second  highest  portion  of  visitations 
in  Year  Three. 

Participation  in  terms  of  the  four  categories  of  indicators  more  than  doubled  in  Year  Two  (Table  5), 
and  increased  marginally  again  in  the  last  year.  Whereas  planning  and  preparation  have 
consistently  been  the  focus  of  close  to  half  of  the  visitations,  the  pupil-teacher  relationships  area 
has  been  receiving  ever-increasing  attention,  nearly  doubling  in  the  three  year  period.  This  may  be 
due  in  part  to  heightened  awareness  of  the  importance  of  pupil-teacher  relationships  to  the  teaching- 
learning  process. 


Table  5 

Purpose  of  Visitations 


Indicators  Number  of  Contacts  Percent  of  Contacts 

1989-90  1990-91  1991-92  1989-90  1990-91  1991-92 


Planning  & 
Preparation 

95 

237 

276 

43.6 

43.7 

44.6 

Instructional 

Strategies 

68 

142 

134 

31.2 

26.2 

21.6 

Communication 

SkiUs 

34 

86 

96 

15.6 

15.9 

15.5 

Pupil-Teacher 

Relationships 

21 

77 

113 

9.6 

14.2 

18.3 

Total 

218 

542 

619 

100.0 

100.0 

100.0 
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The  focus  on  instructional  strategies  has  consistently  declined  since  the  first  year.  The  teacher 
performance  baseline  data,  elaborated  upon  in  a subsequent  section  in  this  report,  identified  three 
specific  behaviors  related  to  instraction  which  were  given  some  of  the  lowest  ratings.  It  is  not 
surprising,  however,  that  these  behaviors  have  not  been  the  subject  of  interclassroom  visitation. 
An  entire  professional  development  day  was  devoted  to  familiarizing  teachers  with  strategies  to 
address  the  specific  concerns  expressed  in  this  area. 

The  majority  of  the  visitations  were  between  colleagues  in  the  same  school.  Table  6 indicates 
that  interschool  visits  accounted  for  approximately  10  percent  of  the  total.  The  majority  of  the 
visitations  and  contacts  teachers  engaged  in  were  outside  of  allotted  EQI  time.  This  may  have 
resulted  fi*om  a number  of  reasons  including  the  unavailability  of  substitute  teachers,  discomfort 
with  using  substitute  teachers  or  preparing  for  them,  scheduling  conflicts,  a desire  not  to  spend 
time  away  from  the  classroom,  or  lack  of  knowledge  regarding  how  to  access  this  time.  More 
frequently  teachers  used  their  personal  preparation  time  to  conduct  visits. 

An  interesting  observation  on  some  of  the  out-of-school  visitations  was  that  several  teachers  were 
permitted  to  visit  schools  in  other  school  jurisdictions  considerable  distances  away.  Though  not 
part  of  the  original  intent  of  the  extra  funding,  this  flexibility  was  gratefully  appreciated  by  the 
teachers  involved. 


Table  6 


Types  of  EQI  Visitations  (Percent) 


Year 

In  School 

Out  of  School 

Allotted  Time 

Outside 
Allotted  Time 

1989-90 

86.6 

13.4 

41.8 

58.2 

1990-91 

88.3 

11.7 

33.2 

66.8 

1991-92 

90.4 

9.6 

38.4 

61.6 

In  School  - Visitations  in  the  same  school  during  school  time. 

Out  of  School  - Collegial  discussion  in  the  same  school  outside  of  school  time. 
Allotted  Time  - Visitations  or  discussion  using  time  allotted  by  the  board. 

Outside  Allotted  Time  - Visitations  or  discussion  outside  of  time  allotted  by  the  board. 
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Mcxiification  of  teaching  behaviors  is,  at  least  in  part,  the  result  of  knowledge  and/or  skills  gained 
through  experience,  through  research  conducted  by  others,  and  through  inservice.  Professional 
development  activities  such  as  teachers'  conventions,  formal  inservice  sessions,  and  this  EQI 
project  provided  opportunities  for  professional  growth. 

Table  7 indicates  a significant  increase  in  the  number  of  teachers  who  attended  out  of  division 
professional  development  activities  since  the  first  year  statistics  were  compiled.  Similarly, 
voluntary  participation  in  the  EQI  project  has  shown  constant  growth.  The  vast  majority  of 
respondents  indicated  this  EQI  project  will  assist  them  in  their  professional  growth.  Teacher 
awareness  of  the  importance  of  professional  development  has  been  enhanced  by  their  involvement 
in  this  project. 


Table  7 


Staff  Survey  Results^ 


Question  1989-90  1990-91  1991-92 

(n=60)  (n=56)  (n=60) 


Yes 

No 

Yes 

No 

Yes 

No 

Attended  professional 
development  out  of  division 

24 

36 

35 

21 

31 

29 

Participated  in  EQI 

46 

14 

49 

7 

52 

8 

Read  materials  to  clarify 
evaluation  indicators 

34 

26 

35 

21 

32 

28 

Feel  EQI  will  assist 
professional  growth  ^ 

48 

7 

44 

4 

45 

6 

1 Results  reported  by  number  of  respondents. 

2 Not  all  teachers  answered  this  question. 
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Teacher  Performance 


Teacher  performance  on  each  of  the  teaching  behaviors  was  rated  on  a five  point  scale.  Table  8 
presents  data  on  the  26  performance  criteria  which  formed  our  baseline.  The  findings  determined 
professional  development  activities.  The  seven  areas  with  the  lowest  mean  rating  were  the  focus  of 
a variety  of  professional  development  activities  in  the  third  year  of  the  project  (see  Appendix  G). 
The  seven  areas  were: 

• 1.6  The  teacher  assures  that  objectives  are  stated  and  understood. 

• 1.9  The  teacher  plans  for  interdisciplinary  activities. 

• 2.1  The  teacher  circulates  during  pupil  activities. 

• 2.5  The  teacher  provides  a physical  environment  which  is  stimulating, 

enriched,  relevant  and  has  displays  of  student  work. 

• 2.6  The  teacher  exhibits  a variety  of  questioning  techniques. 

• 3.2  The  teacher  exhibits  a variety  of  positive  reinforcement  techniques. 

• 3.4  The  teacher  provides  timely  and  relevant  reviews. 

Table  9 provides  data  for  the  second  year  of  observation.  Ratings  changed,  either  as  a result  of 
professional  development  activities,  or  greater  understanding  of  the  indicators. 

With  one  exception,  mean  ratings  have  either  remained  constant  or  have  increased.  More 
important,  the  most  frequent  rating  on  19  of  the  26  teaching  behaviors  was  "5".  This  is  a 
significant  improvement  upon  that  of  the  first  year  in  which  only  seven  received  a modal  rating 
of  *'5".  Of  the  seven  criteria  that  were  the  focus  of  specific  professional  development  activities, 
the  mean  rating  of  three  increased  by  approximately  one-half  point;  no  significant  change  in  the 
mean  rating  was  observed  in  the  other  four.  Three  modal  scores  increased  from  '*4"  to  "5”. 

A comparison  of  the  ratings  suggests  that  there  has  been  considerable  modification  of  teaching 
strategies.  Teacher  awareness  of  the  criteria  used  in  both  the  formative  and  summative  evaluation 
processes  has  contributed  to  this. 
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Table  8 


1991  Teacher  Performance  Baseline^ 


Criteria^  Performance  Rating  Scaled 

1 2 3 4 5 Not  Mean  Mode 

Observed^ 


Sequentially  Developed  Planning  and  Preparation 


1.1  plans 

0 

2 

5 

43 

29 

3 

4.3 

4 

1.2  curriculum 

0 

2 

3 

41 

34 

2 

4.3 

4 

1.3  environment 

0 

0 

10 

38 

31 

3 

4.3 

4 

1.4  individual  needs 

0 

0 

8 

22 

20 

32 

4.2 

4 

1.5  variety 

0 

3 

13 

38 

20 

8 

4.0 

4 

1.6  objectives 

0 

4 

18 

32 

25 

3 

4.0 

4 

1.7  motivation 

0 

1 

18 

34 

26 

3 

4.0 

4 

1.8  relevance 

0 

1 

14 

38 

20 

9 

4.1 

.4 

1.9  interdisciplinary 

0 

5 

11 

21 

14 

31 

3.9 

4 

Key  Instructional 

Strategies 

2.1  circulation 

6 

6 

15 

24 

26 

5 

3.8 

5 

2.2  clarity 

0 

1 

15 

34 

26 

6 

4.1 

4 

2.3  management 

0 

4 

11 

32 

35 

0 

4.2 

5 

2.4  atmosphere 

3 

3 

8 

34 

37 

0 

4.3 

5 

2.5  environment 

1 

3 

16 

38 

21 

3 

3.9 

4 

2.6  questioning 

0 

5 

20 

31 

14 

12 

3.9 

4 

Communication  Skills 

3.1  explanation 

0 

6 

11 

35 

28 

2 

4.1 

4 

3.2  reinforcement 

3 

8 

20 

33 

15 

3 

3.6 

4 

3.3  listening 

0 

1 

15 

36 

30 

0 

4.2 

4 

3.4  review 

0 

1 

13 

33 

11 

24 

3.9 

4 

3.5  language 

0 

0 

15 

36 

31 

0 

4.2 

4 

3.6  parents 

0 

2 

13 

16 

17 

34 

4.0 

5 

Pupil/Teacher  Relationships 

4.1  respect 

0 

1 

8 

31 

42 

0 

4.4 

5 

4.2  expectations 

0 

2 

14 

32 

34 

0 

4.2 

5 

4.3  interaction 

0 

0 

11 

35 

25 

11 

4.2 

4 

4.4  evaluation 

0 

3 

16 

22 

24 

17 

4.0 

5 

4.5  feedback 

0 

1 

9 

24 

18 

30 

4.1 

4 

1 Eighty-two  of  the  87  teachers  (94%)  in  the  Spirit  River  School  Division  were  observed  in  May  1991.  Five 
teachers  were  absent  from  their  duties  during  the  week  the  observations  were  conducted. 

2 The  criteria  are  presented  in  Appendix  D. 

3 Performance  Rating  Scale:  1 - very  poorly  addressed;  2 - poor;  3 - fair;  4 - good;  5 - excellent. 

4 Behaviors  that  were  not  observed  have  been  excluded  in  calculating  the  mean. 
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Table  9 


1992  Teacher  Performance! 


Criteria^ 

Performance  Rating  Scaled 

1 

2 

3 

4 

5 

Not 

Mean 

Mode 

Observed^ 

Sequentially  Developed  Planning  and  Preparation 

1.1  plans 

0 

2 

11 

24 

36 

1 

4.3 

5 

1.2  curriculum 

0 

0 

2 

11 

60 

1 

4.8 

5 

1.3  environment 

0 

1 

2 

32 

39 

0 

4.5 

5 

1.4  individual  needs 

0 

4 

12 

27 

28 

3 

4.1 

5 

1.5  variety 

0 

12 

11 

28 

33 

0 

4.2 

5 

1.6  objectives 

1 

3 

15 

27 

28 

0 

4.0 

5 

1.7  motivation 

0 

1 

11 

32 

30 

0 

4.2 

4 

1.8  relevance 

0 

1 

11 

29 

33 

0 

4.3 

5 

1.9  interdisciplinary 

2 

6 

11 

26 

22 

7 

3.9 

4 

Key  Instructional 

Strategies 

2.1  circulation 

1 

0 

9 

21 

38 

5 

4.4 

5 

2.2  clarity 

0 

1 

4 

35 

33 

1 

4.4 

4 

2.3  management 

0 

1 

11 

29 

33 

0 

4.3 

5 

2.4  atmosphere 

0 

2 

7 

27 

38 

0 

4.4 

5 

2.5  environment 

2 

3 

15 

23 

29 

2 

4.0 

5 

2.6  questioning 

1 

1 

15 

34 

17 

6 

4.0 

4 

Communication  Skills 

3.1  explanation 

1 

2 

13 

27 

31 

0 

4.1 

5 

3.2  reinforcement 

0 

4 

12 

32 

26 

0 

4.1 

4 

3.3  listening 

0 

2 

8 

29 

34 

1 

4.3 

5 

3.4  review 

0 

0 

10 

27 

31 

6 

4.3 

5 

3.5  language 

0 

0 

1 

26 

45 

2 

4.6 

5 

3.6  parents 

0 

3 

17 

29 

24 

1 

4.0 

4 

Pupil/Teacher  Relationships 

4.1  respect 

1 

0 

8 

29 

36 

0 

4.3 

5 

4.2  expectations 

1 

3 

9 

28 

33 

0 

4.2 

5 

4.3  interaction 

0 

2 

9 

25 

35 

3 

4.3 

5 

4.4  evaluation 

0 

1 

8 

33 

32 

0 

4.3 

4 

4.5  feedback 

0 

1 

8 

26 

35 

4 

4.4 

5 

1 Seventy-four  of  the  86  teachers  (86%)  in  the  Spirit  River  School  Division  were  observed  in  May  1992.  Twelve 
teachers  were  absent  from  their  duties  during  the  week  the  observations  were  conducted. 

2 The  criteria  are  presented  in  Appendix  D. 

3 Performance  Rating  Scale:  1 - very  poorly  addressed;  2 - poor;  3 - fair;  4 - good;  5 - excellent. 

4 Behaviors  which  were  not  observed  have  been  excluded  in  calculating  the  mean. 
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Parental  and  Student  Attitudes 


All  students  in  grades  9 through  12  as  well  as  all  parents  of  students  in  grades  2,  5,  8 and  1 1 were 
surveyed  on  their  perception  of  the  validity  of  the  chosen  criteria  for  the  formative  and  summative 
evaluation  of  teachers  (see  Appendix  H). 

In  all  three  years  there  was  generally  reasonable  consistency  in  responses  of  parents  and  students. 
Not  surprisingly  the  vast  majority  of  parents  agreed  that: 

• the  teacher  must  be  an  active  listener 

• explanations  of  objectives,  assignments,  and  behavior  must  be  clear 

• the  language  used  must  be  appropriate  to  the  developmental  level  of  students 

• a safe  environment  with  necessary  resources  should  be  provided 

• individual  learning  needs  must  be  addressed 

• classrooms  must  be  appropriately  managed 

• there  must  be  mutual  respect,  clear  expectations 

• there  must  be  fair  and  meaningful  evaluation  of  student  progress. 

Student  opinion  on  any  of  the  criteria  was  not  as  strong  as  that  of  parents.  Without  exception  a 
significant  proportion  of  students  indicated  'less  important"  as  their  rating  of  any  particular 
criterion. 

Overall,  however,  the  only  significant  disagreement  between  parents  and  students  was  regarding 
the  flow  of  information  to  the  home  and  the  importance  of  the  teacher  circulating  in  the  classroom. 
Students  attached  much  less  importance  to  these  than  did  parents. 

In  general,  parents  indicated  a high  level  of  support  for  the  criteria  being  used  to  assess  teacher 
performance.  This  may  be  interpreted  as  support  for  the  view  that  it  is  these  criteria  which  have  an 
impact  on  student  outcomes. 
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Student  Outcomes 


Cognitive  Outcomes 

Alberta  Education  issues  three  types  of  high  school  diplomas,  and  a certificate:  the  General  High 
School  Diploma,  the  Advanced  High  School  Diploma,  the  High  School  Equivalency  Diploma,  and 
the  Certificate  of  Achievement  for  students  enrolled  in  the  Integrated  Occupational  Program.  The 
diplomas  and  certificate  certify  the  holder  has  successfully  completed  a prescribed  program  of 
instruction.  In  the  Guide  to  Education,  Senior  High  School  Handbook,  published  by  Alberta 
Education,  the  requirements  are  outlined  for  the  diplomas  and  certificate.  All  grade  twelve  students 
who  are  enrolled  in  diploma  subjects  write  provincial  departmental  exams  in  January  or  June, 
depending  on  their  timetables  in  their  schools.  Their  final  mark  is  a blend  of  what  they  have 
obtained  from  fifty  percent  of  the  school  awarded  mark  and  fifty  percent  from  the  diploma  exam. 


Provincial  Diploma  Examinations 

Table  10  displays  the  comparisons  between  provincial  and  Spirit  River  School  Division  students’ 
results  on  diploma  exams  in  June  1990, 1991,  and  1992  with  respect  to  the  percentage  of  students 
achieving  ’’acceptable  standards",  that  is,  achieving  50%  or  higher  on  the  final  blended  mark. 

The  one  significant  difference  between  jurisdiction  and  provincial  results  is  in  mathematics  in 
1991  and  1992.  An  analysis  of  the  results  in  mathematics  suggests  our  students  are  experiencing 
difficulty  applying  procedural  skills.  We  will  address  concerns  about  these  results  in  the  coming 
year.  The  validity  of  comparisons,  however,  is  questionable  because  of  the  low  number  of 
students  involved. 


37 


Table  10 


Percentage  of  Grade  12  Students  Achieving  the  Acceptable  Standard 
or  Higher  on  the  Diploma  Exams  (Final  Blended  Mark) 


June  1990  June  1991  June  1992 

Division  Province  Division  Province  Division  Province 


Social  Studies  30 

100.0 

(n=8) 

89.1 

89.7 

(n=58) 

91.9 

89.5 

(n=19) 

90.0 

Mathematics  30 

100.0 

(n=18) 

82.8 

50.0 

(n=8) 

80.9 

57.1 

(n=21) 

82.5 

Biology  30 

88.9 

(n=25) 

84.9 

96.6 

(n=25) 

88.4 

78.9 

(n=38) 

86.1 

English  30 

100.0 

(n=18) 

94.9 

100.0 

(n=15) 

96.0 

94.4 

(n=18) 

95,3 

English  33 

92.9 

(n=14) 

90.8 

100.0 

<n=18) 

93.0 

100.0 

(n=12) 

92.1 

Physics  30 

100.0 

(n=10) 

90.7 

92.9 

(n=14) 

91.1 

0.0 

(n=l) 

90.2 

Chemistry  30 

85.7 

(n=35) 

86.1 

100.0 

(n=4) 

88.2 

94.1 

(n=34) 

89.5 

The  acceptable  standard  is  50%  or  higher  on  the  final  blended  mark. 
Source:  Alberta  Education 


38 


Provincial  Achievement  Tests 

Provincial  achievement  tests  are  given  to  students  in  grades  3, 6 and  9.  Comparisons  between 
provincial  and  jurisdiction  results  on  achievement  tests  are  displayed  in  Table  1 1.  Generally, 
jurisdiction  results  are  within  the  provincial  range  in  all  three  years. 


Table  11 

Percentage  of  Grade  3,  6 and  9 Students  Achieving  the  Acceptable 
Standard  or  Higher  on  the  Total  Provincial  Achievement  Tests 


Division 

1990 

Province 

Mathematics  3 

94.0 

(n=100) 

91.8 

Science  6 

76.2 

(n=84) 

82.5 

Language  Arts  9 

81.6 

(n=87) 

82.9 

Division 

1991 

Province 

Science  3 

82.4 

(n=88) 

78.9 

Mathematics  6 

75.0 

(n=100) 

75.5 

Social  Studies  9 

80.2 

(n=91) 

78.9 

Division 

1992 

Province 

Social  Studies  3 

80.0 

(n=90) 

83.5 

Language  Arts  6 

75.3 

(n=93) 

75.6 

Mathematics  9 

63.7 

(n=80) 

67.4 

Source:  Alberta  Education 
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Affective  Outcomes 

The  Student's  Perception  of  Ability  Scale  was  administered  to  all  grade  2 students  and  the 
Classroom  Environment  Scale  to  all  students  in  grades  5,  8 and  1 1 in  the  division.  Both 
instruments  are  designed  to  yield  information  on  student  attitudes  toward  various  aspects 
of  the  school  environment. 

Divisional  mean  scores  for  1991  and  1992,  as  well  as  normed  means  for  the  Student's 
Perception  of  Ability  Scale  are  presented  in  Table  12.  Divisional  mean  scores  on  all  dimensions 
are  consistently  above  the  normed  mean  in  both  years. 

Similarly  Table  13  and  Table  14  illustrate  the  results  for  the  Classroom  Environment  Scale 
administered  in  language  arts  (English)  and  mathematics  classes  respectively.  With  statistics 
having  been  compiled  for  only  two  years,  it  is  premature  to  draw  any  conclusions  at  this  point. 
Patterns  may  begin  to  emerge  after  five  to  six  years. 


Table  12 

Student's  Perception  of  Ability  Scale  Results 


Normative  Sample 

Divisional  Mean 

Mean 

SDl 

1991 

1992 

Scales 

(n=642) 

(n=105) 

(n=102) 

Full  Scale 

46.24 

11.71 

50.45 

50.91 

General  Ability 

7.91 

3.01 

8.64 

8.22 

Arithmetic 

9.17 

3.01 

10.15 

9.77 

School  Satisfaction 

7.99 

2.78 

8.45 

8.77 

Reading/Spelling 

9.07 

3.13 

9.48 

9.39 

Penmanship/Neamess 

7.89 

3.00 

8.78 

9.18 

Confidence 

4.21 

2.25 

4.95 

5.48 

1 Standard  Deviation 
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Table  13 


Classroom  Environment  Scale  Results  for  Language  Arts  Classes 


Normative  Sample  Divisional  Mean 


Subscales^ 

Mean  SD^ 

(n=92  classes) 

1991 

(n=21)3 

1992 

(n=21)3 

Involvement 

4.70 

1.62 

5.70 

5.83 

Affiliation 

6.13 

1.03 

6.54 

6.21 

Teacher  Support 

6.70 

1.60 

6.08 

5.99 

Task  Orientation 

5.44 

1.73 

6.53 

6.31 

Competition 

5.14 

1.19 

6.53 

6.12 

Order  & Organization 

5.44 

1.79 

5.40 

5.57 

Rule  Clarity 

5.42 

1.32 

6.51 

6.19 

Teacher  Control 

3.61 

1.66 

5.33 

5.68 

Innovation 

5.17 

1.81 

5.20 

5.05 

1 Descriptions  of  dimensions  are  presented  in  Appendix  E. 

2 Standard  deviation. 

3 Number  of  classes. 


Table  14 

Classroom  Environment  Scale  Results  for  Mathematics  Classes 


Normative  Sample 

Divisional  Mean 

Mean 

SD2 

1991 

1992 

Subscales^ 

(n=48  classes) 

(n=21)3 

(n=2i: 

Involvement 

4.37 

1.26 

5.48 

5.67 

Affiliation 

6.07 

1.09 

6.74 

6.15 

Teacher  Support 

6.08 

1.45 

6.28 

5.86 

Task  Orientation 

7.32 

1.58 

7.38 

6.99 

Competition 

5.38 

0.88 

6.60 

6.51 

Order  & Organization 

6.09 

1.99 

5.33 

5.52 

Rule  Clarity 

6.17 

1.28 

6.72 

6.32 

Teacher  Control 

4.34 

1.72 

6.20 

6.58 

Innovation 

3.65 

1.16 

4.32 

4.36 

1 Descriptions  of  dimensions  are  presented  in  Appendix  E. 

2 Standard  deviation. 

3 Number  of  classes. 


41 


Discipline 

Discipline  outcomes,  displayed  in  Table  15,  are  to  an  extent  a reflection  of  student  attitude  toward 
school.  The  students'  perception  of  the  school  environment  is  one  factor  which  does  have  an 
impact  on  their  behavior  in  school. 

No  discernible  pattern  is  emerging  in  the  statistics  compiled  to  date.  In  Year  Two,  the  numbers 
of  disciplinary  measures  taken  dropped  considerably  from  the  previous  year.  In  the  1991-92 
school  year  these  numbers  increased  substantially.  This  may  be  attributable  to  changes  in 
administration  in  three  of  our  eight  schools  and/or  changes  in  discipline  procedures  throughout 
the  division.  Dropouts  are  those  students  who  register,  but  do  not  complete  their  full  year  of 
schooling.  Dropouts  though  tabulated  were  not  considered  in  this  study. 

At  this  time  it  is  not  possible,  nor  would  it  be  appropriate,  to  attempt  to  assess  the  effectiveness  of 
modifying  instructional  strategies  and  the  impact  of  this  modification  on  student  outcomes. 


Table  15 

Discipline  Outcomes 


Year 

Referrals 

Suspensions 

Expulsions 

Dropouts 

1989-90 

835 

110 

0 

14 

1990-91 

560 

49 

0 

12 

1991-92 

1,080 

112 

1 

16 

Summary  of  Results 

In  general,  our  achievement  and  diploma  exam  results  fall  within  the  acceptable  range,  some  being 
above  the  provincial  mean,  others  below.  No  obvious  patterns  are  emerging.  Similarly,  the 
information  we  have  gathered  on  affective  outcomes  will  only  become  of  value  when  used  for 
comparison  over  an  extended  period  of  time.  In  order  that  we  can  appropriately  assess  the  results 
and  establish  correlations,  additional  years  of  data  collection  and  documentation  are  required.  To 
terminate  the  process  at  this  point  may  in  fact  be  counterproductive.  It  is,  therefore,  our  intent  to 
continue  with  the  data  collection. 
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Chapter  V 


SUMMARY  AND  DISCUSSION 

This  chapter  briefly  outlines  the  purpose  of  the  study,  the  findings,  and  conclusions  and 
implications  for  practice  and  further  research.  Recommendations  and  plans  for  the  future 
of  this  project  in  the  Spirit  River  School  Division  are  also  addressed. 


Purpose 

A wealth  of  information  on  effective  teaching  behaviors  has  emerged  over  the  last  few  decades. 
This  specific  study  involved  identification  and  validation  of  a set  of  indicators  that  could  be 
used  to  assess  teacher  effectiveness  in  formative  and  summative  processes.  The  focus  was  on 
the  development  of  criteria  for  teacher  effectiveness.  The  question  posed  was:  ”Do  the  26  teaching 
behaviors  our  professional  staff  identified  as  being  critical  to  effective  teaching  have  an  impact 
on  student  outcomes  in  both  the  cognitive  and  affective  domains?"  We  are  currently  establishing 
normed  measures  of  teacher  behaviors  and  measuring  student  cognitive  and  affective  outcomes 
as  they  relate  to  those  behaviors.  The  effectiveness  of  modifying  instructional  strategies  will  be 
monitored  over  a period  of  time.  There  will  be  a need  to  assess  the  results  of  the  intervention  as 
it  applies  to  student  outcomes,  should  the  results  support  this. 

Findings 

The  active  involvement  of  professional  staff  in  action  research  is  the  most  positive  aspect  of  our 
EQI  project  Table  7 clearly  illustrates  the  value  teachers  have  attached  to  this  project  in  terms  of 
it  assisting  them  with  their  professional  growth.  Only  about  10  percent  of  all  teacher  responding 
to  the  survey  felt  otherwise  and  consequently  would  not  have  participated  voluntarily. 
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Direct  staff  input  into  the  development  and  validation  of  a set  of  indicators  which  can  be  used  to 
assess  teacher  effectiveness  in  both  formative  and  summative  processes  provided  a sense  of 
ownership  and  subsequently  enhanced  the  likelihood  of  participation.  The  number  of  classroom 
visitations  made  each  year  is  indicative  of  teachers'  willingness  to  be  active  participants  in  this 
process. 

There  is  considerable  evidence  to  support  the  position  that  teachers  should  play  a significant  role 
in  setting  the  criteria  by  which  their  performance  is  judged.  Teachers  in  this  jurisdiction  did  set  the 
criteria  to  be  used  in  the  formative  evaluation  process,  and  subsequently  supported  adoption  of 
these  same  criteria  for  the  summative  process.  Ownership,  once  again,  had  a positive  impact  upon 
teacher  willingness  to  accept  these  as  legitimate  criteria. 

Teachers  have  shown  a willingness  to  modify  teaching  behaviors.  Tables  8 and  9 illustrate  a 
change  in  the  modal  rating  on  a considerable  number  of  criteria.  The  mean  rating  of  a number 
of  criteria  has  also  improved.  Clear  descriptions  of  the  meaning  of  each  criterion  results  from 
discussions  with  colleagues  and  administrators  and  may  have  contributed  to  this  change. 

Teachers  have  actively  participated  in  the  review  and  major  revisions  of  our  student  and  school 
evaluation  policies  and  processes  to  bring  them  "in  line"  with  the  indicators  we  have  adopted. 

The  questionnaire  completed  by  teachers  included  provisions  for  written  comments  regarding  the 
EQI  project.  Strong  support  is  evident  particularly  as  it  relates  to  professional  growth.  The 
following  comments  typify  teacher  perceptions: 

• "A  good  project  that  promotes  and  facilitates  teacher  development" 

• "I  think  the  EQI  project  is  very  valuable  to  all  teachers.  It  gives  a good  opportunity  to 
observe  others  in  action  as  well  as  to  receive  input  to  help  enhance  our  classroom 
environment." 

Our  project  is  exemplary  in  its  involvement  of  all  professional  staff.  Such  extensive  involvement  of 
all  stakeholders,  including  students  and  parents,  has  the  potential  of  paying  dividends  in  terms  of 
student  academic  and  social  outcomes. 

1 . With  very  few  exceptions  parents  generally  indicated  a high  level  of  support  for  the 
criteria  currently  being  used  to  judge  teacher  performance. 
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2.  A safe  and  prcxiuctive  environment,  attention  to  individual  learning  needs,  clarity 
of  objectives,  the  teacher  being  an  active  listener  and  the  entire  group  of  pupil/teacher 
relationships  are  the  highest  priorities  of  parents.  It  is  these  factors  which  would  have  the 
greatest  impact  on  student  attitudes  toward  school  and  subsequently  on  performance. 


Conclusions 

At  this  point  in  time  it  is  premature  to  speculate  on  the  impact  modification  of  instmctional 
strategies  is  having  upon  student  outcomes.  As  a result,  only  conclusions  relating  to  strategies 
that  have  been  successful  in  promoting  the  active  involvement  of  professional  staff  in  this  process 
can  be  shared.  Other  jurisdictions  may  see  potential  value  for  incorporating  components  of  these 
strategies  into  plans  of  action  they  may  be  pursuing. 

With  the  support  and  involvement  of  staff,  we  believe  the  project  is  proving  to  have  a very  positive 
and  professionally  rewarding  impact  on  teaching  skills  for  the  benefit  of  students.  It  is  the  impact 
on  student  outcomes  that  will  continue  to  be  the  focus  of  this  study. 


Implications  and  Recommendations 

Based  on  the  findings  of  this  study,  the  following  implications  were  derived.  Reference  is  made 
exclusively  to  teachers  as  they  were  to  "key  players"  in  this  process. 

Staff  support  is  crucial  to  the  success  of  any  intervention  strategy.  Without  the  voluntary  support 
of  teachers,  a study  of  this  nature  and  magnitude  would  not  have  been  possible.  Mandates  were  not 
a part  of  this  process.  Staffs  must  be  given  the  opportunity  for  voluntary  involvement  in  the  very 
early  stages  of  any  project  or  study  similar  to  this.  This  enhances  the  likelihood  of  acceptance  and 
participation  over  an  extended  period  of  time. 

Teachers  must  have  the  opportunity  and  the  channels  for  "feeding  " into  the  system.  This 
generates  an  allegiance  to  the  system  of  which  they  are  a part.  Under  the  leadership  of  central 
office  administration  all  decisions  relating  to  this  project  were  made  by  a local  committee  of 
teachers.  This  included  the  identification  of  indicators  of  effective  teaching,  formulation  of  key 
variables  which  could  impact  the  effectiveness  of  implementation,  identification  of  26  teaching 
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behaviors  perceived  to  be  critical  to  positive  student  outcomes,  articulation  of  clear  explanations  of 
objectives,  development  and  selection  of  data  gathering  instruments,  and  the  development  of  an 
action  plan  for  implementation  in  all  division  schools. 

Direction  and  support  from  division  administrators  and  the  school  board  is  critical  to  the  potential 
for  significant  changes  being  made.  The  school  board's  commitment  to  this  project,  evident  in  the 
funding  it  provided  to  cover  the  cost  of  substitute  teachers,  the  addition  of  a staffing  component  to 
all  small  schools,  and  the  priority  given  the  project  in  the  divisions  Three  Year  Plan  were  perceived 
as  support  for  the  project  and  its  intent.  This  has  provided  the  time  teachers  need  for  planning, 
introduction  and  implementation.  Tangible  benefits  or  results  enhance  the  likelihood  of  continued 
commitment  and  participation.  Rewards,  whether  intrinsic  or  extrinsic,  serve  as  motivational  , 
tools. 


Project  Follow-up 

The  Board  of  Trustees  of  the  Spirit  River  School  Division,  with  the  support  of  senior  administration, 
has  made  a commitment  to  carry  on  with  the  study.  Funding  has  been  budgeted  to  cover  costs  of 
substitute  teachers  and  the  additional  staffing  component  in  our  small  schools  for  the  forthcoming 
school  year. 

The  extensive  data  collection  currently  in  place  will  be  continued.  We  are  now  completing  only  our 
first  measurements  of  the  effectiveness  of  modifying  instructional  strategies.  In  order  that  we  may 
appropriately  assess  the  results  and  establish  relationships,  should  the  results  support  this, 
additional  years  of  research  and  documentation  are  required.  To  stop  the  process  at  this  point 
would  have  a negative  impact  on  the  accomplishments  of  the  last  three  years. 

It  is  also  our  intent  to  continue  to  inform  our  publics  of  the  efforts  being  directed  at  making  our 
schools  a better  place  for  students.  The  local  media  and  our  school  newsletters  will  continue  to 
serve  as  the  primary  disseminators  of  information  to  the  public.  Discussions,  productions  and 
presentation  at  special  events  such  as  Education  Week  will  continue  to  augment  this  process. 

Our  ability  to  demonstrate  to  our  publics  that  education  is  an  excellent  investment  in  our  children 
and  our  future  will  continue  to  be  a high  priority. 
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Concluding  Statement 


Teachers  do  make  a difference!  This  EQI  project  has  been  a positive  experience  for  many 
stakeholders  in  the  Spirit  River  School  Division.  We  believe  that  the  project  is  already 
having  a very  strong  and  professionally  rewarding  impact  on  teaching  skills  for  the  benefit 
of  our  students. 

Using  the  same  criteria  for  both  summative  and  formative  evaluations  has  heightened  teacher 
awareness  regarding  professional  expectations.  The  clear  focus  on  learner  needs  and  student 
outcomes  and  the  staff  development  opportunities  that  are  being  provided  are  procedures  that  could 
be  employed  effectively  in  any  district 

Similarly,  the  three  documents  prepared  in  the  course  of  our  own  involvement  in  this  project, 
the  Student  Evaluation  Handbook,  the  School  Improvement  Handbook,  and  the  Teacher  Planbook, 
are  all  germane  to  promoting  instruction  in  the  classroom. 
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Appendix  A 

Working  Documents  For  Year  One 


Purpose  and  Intent 

To  establish  indicators  of  effective,  quality  teaching,  in  a collaborative  formative  process,  on  a 
voluntary  and  anonymous  basis,  determine  their  validity  and  impact  as  to  whether  these  indicators 
make  a real  difference  to  student  learning. 


Getting  Started 

1 . Use  one  of  the  self-evaluation  surveys  to  zero  in  on  an  area  you  would  like  to  clarify  or  verify 
in  terms  of  your  own  teaching.  Copies  are  available  from  your  principal  or  the  Instructional 
Media  Centre. 

2 . Seek  collegial  assistance. 

3 . Record  your  data  on  your  school  chart. 

4 . Take  advantage  of  any  opportunities  to  continue  to  seek  collegial  assistance,  within  your 
your  school,  within  the  school  division,  or  further  afield.  Please  record  all  professional 
contacts  on  your  school  chart. 

5 . Share  your  thoughts  on  the  indicators  or  the  process  with  any  of  the  Local  Committee 
members,  so  we  can  plan  revisions  for  next  year. 


Variables 

Variables  are  factors  that  will  have  an  impact  on  any  project  through  outside  our  external  impact. 

1 . Geographic  Distance,  Spareness  and  Travel  Time 

Evidence:  Keep  track  of  contact  with  other  colleagues  regarding  self-improvement 

2 . Extra  time  Provided  for  Staff  Formative  Evaluation  Process 

Evidence:  Keep  track  of  how  allocated  time  is  being  used  for  project  objectives. 

3.  Strong  Inservice/Professional  Development 

Evidence:  Keep  track  of  the  number  of  teachers  attending  EQI  workshops  and/or  who 
sign  out  support  documents/resources. 

4 . Teacher  Experience 

Evidence:  Keep  track  of  school-based  numbers  of  teachers  who  participate  in  formative 
exchanges. 
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5 . Staf^Administrative  Attitude 


Evidence:  Keep  track  of  the  number  of  teachers  participating. 


Outcomes 

We  require  data  to  fulfill  the  requirements  of  this  project.  The  following  are  examples  of  data 
gathering  which  will  indicate  whether  this  project  does  make  a difference  to  student  learning 
on  a divisional  basis. 

1 . Staff  self-evaluation  surveys. 

2 . Rate  of  staff  participation  in  the  project 

3 . Divisional  averages  on  standardized  tests. 

4.  Student  interest  surveys. 

5 . Data  gathering  staff  instruments. 

6.  a.  Classroom  disruptions/referrals  to  school  office. 

b.  Suspensions/expulsions 

c.  Dropout  rates 
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Educational  Quality  Indicators 
Record  Sheet 


Please  take  a moment  to  fill  in  your  EQI  contracts  on  your  school  chart  after  each  contract.  You 
may  want  to  keep  a personal  record  in  your  book.  If  more  than  one  puipose/objective  is 
discussed,  please  write  each  number  in  the  space  provided. 


Date  Years  of  experience: 
(Check  one) 

0-2  3-5  6-8  9-10  11+ 


Purpose:  Local: 

Objective  in  school 

number 


Distant:  Was  this 

out  of  allotted 

school  EQI  time? 


Appendix  B 

Parent  and  Student  Surveys 


Letter  to  Parents 


Dear  Parents: 

The  Spirit  River  School  Division  is  now  completing  the  third  and  final  year  of  a 
three  year  Action  Research  Project  (Educational  Quality  Indicators  Project)  in 
cooperation  with  Alberta  Education.  Our  specific  purpose  is  to  determine  what 
are  the  conditions  which  are  the  most  important  for  effective  teaching  and 
instruction.  A local  committee  of  experienced  Divisional  teachers  developed  the 
initial  criteria,  input  of  parents,  students,  and  teachers  was  gathered  through 
surveys  such  as  this  the  last  two  years,  and  the  resulting  set  of  criteria  is  being 
utilized  in  addressing  teacher  performance  in  the  classroom. 

Parents  with  children  in  grades  two,  five,  eight  and  eleven  have  been  selected  to 
give  a parental  cross  section  perspective  to  this  process.  We  would  appreciate 
your  completing  the  following  questionnaire  and  returning  it  in  confidence  to 
your  school  or  the  Division  Office  by  May  10th,  1992.  Your  anonymity  is 
guaranteed.  These  parental  responses  will  be  summarized  and  given  serious 
consideration  in  the  refining  of  our  criteria  for  the  third  and  final  year  of  this 
project. 

Thank  you  for  taking  the  time  to  complete  and  return  this  survey.  Through  your 
efforts  the  process  will  become  just  that  much  more  meaningful. 

Yours  truly. 


Roger  Rymhs 

Deputy  Superintendent  of  Schools 
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As  a parent,  how  important  do  you  think  each  of  these  qualities  is  for  a good 
learning  environment  for  your  child(ren)?  Please  mark  one  box  beside  each 
statement. 

Very  Less  Don't 

Important  Important  Know 


T ■ Sequentially  Developed  Planning  and  Preparation 

1 . 1 The  teacher  maintains/develops  short,  unit  and  long  term, 

flexible  plans  with  an  evaluation  process.  □ □ 

1 .2  The  teacher  adheres  to  Curticulum/Program  of  Studies 

documents.  □ O 


□ 

□ 


1.3 

The  teacher  provides  a safe  and  organized  environment 
with  resources  and  equipment  readily  available. 

□ 

□ 

1.4 

The  teacher  provides  for  individual  learning  needs 
with  consideration  of  student  abilities,  based  on 
diagnostic  information. 

□ 

□ 

1.5 

The  teacher  provides  a variety  of  learning  activities. 

□ 

□ 

1.6 

The  teacher  ensures  that  objectives  are  stated 
and  understood. 

□ 

□ 

1.7 

The  teacher  provides  for  motivation  of  students. 

□ 

□ 

1.8 

The  teacher  makes  learning  activities  relevant. 

□ 

□ 

1.9 

The  teacher  plans  for  interdisciplinary  activities. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 


2.  Key  Instructional  Strategies 

2.1  The  teacher  circulates  during  pupil  activities.  O 

2 . 2 The  teacher  presents  clear  assignments  which 

are  relevant  to  student  experiences.  □ 

2.3  The  teacher  exhibits  classroom  management  skills.  □ 

2 .4  The  teacher  provides  an  environment  whereby 
active  learning  takes  place  within  a working  a 

and  non-threatening  atmosphere.  O 

2 . 5 The  teacher  provides  a physical  classroom 
environment  which  is  stimulating,  enriched,  relevant 

and  has  displays  of  smdent  work.  □ 

2.6  The  teacher  exhibits  a variety  of  questioning  techniques.  O 


□ 

□ 

□ 

□ 

□ 

□ 


□ 

□ 

□ 

□ 

□ 

□ 
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Very 

Important 


Less 

Important 


Don't 

Know 


3.  Communication  Skills 


3 . 1 The  teacher  provides  clear  explanation  of  objectives, 


assignments  and  behavioral  expectations. 

□ 

□ 

□ 

3.2 

The  teacher  exhibits  a variety  of  positive 
reinforcement  techniques. 

□ 

□ 

□ 

3.3 

The  teacher  is  an  active  listener. 

□ 

□ 

□ 

3.4 

The  teacher  provides  timely  and  relevant  reviews. 

□ 

□ 

□ 

3.5 

The  teacher  uses  language  appropriate  to  the 
developmental  level  of  students. 

□ 

□ 

□ 

3.6 

The  teacher  maintains  regular  parental  information  flow. 

□ 

□ 

□ 

4. 

4.1 

Pupil/Teacher  Relationships 

Mumal  respect  is  exhibited  between  teacher 
and  students. 

□ 

□ 

□ 

4.2 

Clear  routines,  expectations  and  rules  are  made  known 
and  followed. 

□ 

□ 

□ 

4.3 

The  teacher  provides  for  positive  peer  interaction/support. 

□ 

□ 

□ 

4.4 

The  teacher  uses  a fair  and  meaningful  evaluation 
process. 

□ 

□ 

□ 

4.5 

The  teacher  provides  prompt  feedback  on  student 
assignments. 

□ 

□ 

□ 
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Apart  from  personal  qualities,  please  add  any  other  measurable  area 
you  feel  is  important  for  effective  teaching. 


Return  to:  Your  School 


or 

Spirit  River  School  Division  #47 
Box  99 

SPIRIT  RIVER,  AB 
TOH  3GO 
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Letter  to  Students 


Dear  Students: 

The  Spirit  River  School  Division  is  now  completing  the  third  and  final  year  of  a 
three  year  Action  Research  Project  (Educational  Quality  Indicators  Project)  in 
cooperation  with  Alberta  Education.  Our  specific  purpose  is  to  determine  what 
are  the  conditions  which  are  the  most  important  for  effective  teaching  and 
instruction.  A local  committee  of  experienced  Divisional  teachers  has  developed 
the  initial  criteria,  which  have  been  used  by  teachers  in  Divisional  classrooms 
during  the  past  two  years. 

Students  in  grades  nine,  ten,  eleven  and  twelve  have  been  selected  once  again  to 
give  a cross  section  perspective  to  this  process.  Accordingly,  would  you  be  good 
enough  to  complete  the  following  questionnaire  and  return  it  in  confidence  to 
your  school  or  the  Division  Office  by  May  10th,  1992.  These  student  responses 
will  be  summarized  and  given  serious  consideration  in  the  refining  of  our  criteria 
for  the  coming  year. 

Thank  you  for  taking  the  time  to  make  this  process  more  meaningful,  since  we 
are  seeking  information  and  help  for  the  Project  from  students,  parents  and 
teachers. 

Yours  truly. 


Roger  Rymhs 

Deputy  Superintendent  of  Schools 
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As  a student,  how  important  do  you  think  each  of  these  qualities  is  for  a good 
learning  environment  for  you?  Please  mark  one  box  beside  each  statement. 


1. 

Sequentially  Develoned  Planning  and  Prenaration 

Very 

Important 

Less 

Important 

Don't 

Know 

1.1 

The  teacher  maintains/develops  short,  unit  and  long  term, 
flexible  plans  with  an  evaluation  process. 

□ 

□ 

□ 

1.2 

The  teacher  adheres  to  Curriculum/Program  of  Studies 
documents. 

□ 

□ 

□ 

1.3 

The  teacher  provides  a safe  and  organized  environment 
with  resources  and  equipment  readily  available. 

□ 

□ 

□ 

1.4 

The  teacher  provides  for  individual  learning  needs 
with  consideration  of  student  abilities,  based  on 
diagnostic  information. 

□ 

□ 

□ 

1.5 

The  teacher  provides  a variety  of  learning  activities. 

□ 

□ 

□ 

1.6 

The  teacher  ensures  that  objectives  are  stated 
and  understood. 

□ 

□ 

□ 

1.7 

The  teacher  provides  for  motivation  of  students. 

□ 

□ 

□ 

1.8 

The  teacher  makes  learning  activities  relevant. 

□ 

□ 

□ 

1.9 

The  teacher  plans  for  interdisciplinary  activities. 

□ 

□ 

□ 

2. 

2.1 

Kev  Instructional  Strategies 

The  teacher  circulates  during  pupil  activities. 

□ 

□ 

□ 

2.2 

The  teacher  presents  clear  assignments  which 
are  relevant  to  student  experiences. 

□ 

□ 

□ 

2.3 

The  teacher  exhibits  classroom  management  skills. 

□ 

□ 

□ 

2.4 

The  teacher  provides  an  environment  whereby 
active  learning  takes  place  within  a working  a 
and  non-threatening  atmosphere. 

□ 

□ 

□ 

2.5 

The  teacher  provides  a physical  classroom 
environment  which  is  stimulating,  enriched,  relevant 
and  has  displays  of  student  work. 

□ 

□ 

□ 

2.6 

The  teacher  exhibits  a variety  of  questioning  techniques. 

□ 

□ 

□ 
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Very 

Important 

Less 

Important 

Don't 

Know 

3. 

Communication  Skills 

3.1 

The  teacher  provides  clear  explanation  of  objectives, 
assignments  and  behavioral  expectations. 

□ 

□ 

□ 

3.2 

The  teacher  exhibits  a variety  of  positive 
reinforcement  techniques. 

□ 

□ 

□ 

3.3 

The  teacher  is  an  active  listener. 

□ 

□ 

□ 

3.4 

The  teacher  provides  timely  and  relevant  reviews. 

□ 

□ 

□ 

3.5 

The  teacher  uses  language  appropriate  to  the 
developmental  level  of  students. 

□ 

□ 

□ 

3.6 

The  teacher  maintains  regular  parental  information  flow. 

□ 

□ 

□ 

4. 

Pupil/Teacher  Relationships 

4.1 

Mutual  respect  is  exhibited  between  teacher 
and  students. 

□ 

□ 

□ 

4.2  Clear  routines,  expectations  and  rules  are  made  known 


and  followed. 

□ 

□ 

□ 

4.3 

The  teacher  provides  for  positive  peer  interaction/support. 

□ 

□ 

□ 

4.4 

The  teacher  uses  a fair  and  meaningful  evaluation 
process. 

□ 

□ 

□ 

4.5 

The  teacher  provides  prompt  feedback  on  student 
assignments. 

□ 

□ 

□ 
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Apart  from  personal  qualities,  please  add  any  other  measurable  area 
you  feel  is  important  for  effective  teaching. 


Return  to:  Your  School 


or 

Spirit  River  School  Division  #47 
Box  99 

SPIRIT  RIVER,  AB 
TOH  3GO 
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Appendix  C 
Staff  Survey 


1 . Did  you  attend  an  out  of  division  professional  development  activity  other  than  the 
March  Teachers’  Convention? 

2 . Did  you  participate  in  the  EQI  Project  this  year? 

3 . If  yes,  approximately  how  many  times? 

4 . Did  you  read  extra  material  to  help  clarify  the  Evaluation  Indicators? 

5 . EQI  will  assist  me  in  my  professional  growth  . . . 

6.  Check  one: 


Teacher 

Administrator 

7 . Please  comment  on  the  EQI  Project. 
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Appendix  D 

Teacher  Performance  Tally  Sheet 


(Scale  ”5'*  represents  the  highest  possible) 

T.  Seaiientiallv  Developed  Planning  and  Preparation 

1 . 1 The  teacher  maintains/develops  short,  unit  and  long  term, 
flexible  plans  with  an  evaluation  process. 

1 . 2 The  teacher  adheres  to  Curriculum/Program  of  Studies 
documents. 

1 . 3 The  teacher  provides  a safe  and  organized  environment 
with  resources  and  equipment  reachly  available. 

1 . 4 The  teacher  provides  for  individual  learning  needs 
with  consideration  of  student  abilities,  bas^  on 
diagnostic  information. 

1 . 5 The  teacher  provides  a variety  of  learning  activities. 

1 . 6 The  teacher  ensures  that  objectives  are  stated 
and  understood. 

1 . 7 The  teacher  provides  for  motivation  of  students. 

1 . 8 The  teacher  makes  learning  activities  relevant 

1 . 9 The  teacher  plans  for  interdisciplinary  activities. 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 


IL Key  Instructional  Strate2ies 

2. 1 The  teacher  circulates  during  pupil  activities. 

2 . 2 The  teacher  presents  clear  assignments  which 
are  relevant  to  student  experiences. 

2 . 3 The  teacher  exhibits  classroom  management  skills. 

2 . 4 The  teacher  provides  an  environment  whereby 
active  learning  takes  place  within  a working  a 
and  non-threatening  atmosphere. 

2 . 5 The  teacher  provides  a physical  classroom 
environment  which  is  stimulating,  enriched,  relevant 
and  has  displays  of  student  work. 

2.6  The  teacher  exhibits  a variety  of  questioning  techniques. 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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III.  Communication  Skills 


3 . 1 The  teacher  provides  clear  explanation  of  objectives, 
assignments  and  behavioral  expectations. 

3 . 2 The  teacher  exhibits  a variety  of  positive 
reinforcement  techniques. 

3 . 3 The  teacher  is  an  active  listener. 

3 . 4 The  teacher  provides  timely  and  relevant  reviews. 

3 . 5 The  teacher  uses  language  appropriate  to  the 
developmental  level  of  students. 

3 . 6 The  teacher  maintains  regular  parental  information  flow. 


DL Pupil/Teacher  Relationships 

4. 1 Mutual  respect  is  exhibited  between  teacher 
and  students. 

4.2  Clear  routines,  expectations  and  mles  are  made  known 
and  followed. 

4 . 3 The  teacher  provides  for  positive  peer  interaction/support. 

4.4  The  teacher  uses  a fair  and  meaningful  evaluation 
process. 

4 . 5 The  teacher  provides  prompt  feedback  on  student 
assignments. 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Appendix  E 

Classroom  Environment  Scale 


Domain: 

processes;  schooling 

Origin: 

United  Stales 

Purpose: 

"to  assess  the  social  climate  of  junior  high  and  high  school  classrooms" 
(authors) 

Description: 

9 scores:  Relationship  dimensions  (involvement,  affiliation,  teacher 
support);  Personal  GrowthlGoal  Orientation  dimensions  (task  orientation, 
competition);  System  Maintenance  ojid  Change  dimensions  (order  and 
organization,  rule  clarity,  teacher  control,  innovation); 

3 forms:  real  (90  items),  ideal  (90  items),  short  (36  items) 

Age/grade  range: 

junior  and  high  school  students;  teachers 

Test  development/ 
publication: 

1974-1987 

Author(s)/availability: 

Rudolf  H.  Moos,  Edison  J.  Trickett; 

Consulting  Psychologists  Press,  Inc.;  Institute  of  Psychological  Research,  Inc. 

Reliability: 

Kuder-Richardson  (n=465;  .67  to  .86) 
test-retest  (6  weeks,  n=52;  .72  to  .90) 

Validity: 

concurrent  (with  teacher  interview  and  observational  data) 
construct  (with  absenteeism,  self-concept,  academic  motivation) 

Norms: 

broad  range  of  classroom  (attempts  to  include  variables  such  as  SES,  size  of 

school,  ethnic  mix,  public^rivate  schools) 

students:  382  classrooms  (average  size  of  20  to  30  students) 

teachers:  295  classrooms 

separate  norms  for  different  subjectsi areas:  science;  math;  business  and 
technical;  English/history/govemment/economics/social  studies 
(combined  as  one  set  of  norms) 

Reviews: 

MMY  10th  ed.  #60  {RA.  Saudargas,  Associate  Professor  of  Psychology, 
University  of  Tennessee,  Knoxville,  TN;  CE.  Smith,  Associate  Professor 
of  Special  Education  and  Rehabilitation,  Syracuse  university,  Syracuse,  NY.) 

Strengths: 

good  norms  (although  one  reviewer  commented  that  original  norms  are  now 
14  years  old);  strong  reliability;  modest  validity;  utilized  in  a large  number 
of  studies  to  date;  the  Short  Form  has  acceptable  technical  properties 

Weaknesses: 

factor  structure  does  not  reproduce  the  3 main  dimensions;  norms  can  be 
broken  down  by  other  variables  such  as  size  of  school 

Usefulness  for  EQI: 

yes 

Note.  From  Educational  Quality  Indicators:  Inventory  of  Assessment  Instruments  ( p.  29)  by  Alberta  Education, 
1990,  Edmonton:  Corporate  and  Fiscal  Planning.  Reprinted  by  permission. 
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CES  Subscale  and  Dimension  Descriptions* 


Relationship  Dimensions 


1 . Involvement 

the  extent  to  which  students  are  attentive  and 
interested  in  class  activities,  participate  in  discussions, 
and  do  additional  work  on  their  own 

2 . Affiliation 

the  level  of  friendship  students  feel  for  each  other, 
as  expressed  by  getting  to  know  each  other, 
helping  each  other  with  homework,  and  enjoying 

3 . Teacher  Support 

the  amount  of  help  and  friendship  the  teacher 
manifests  toward  students;  how  much  the  teacher 
talks  openly  with  students,  trusts  them,  and  is 
interested  in  their  ideas 

Personal  Growth/Goal  Orientation  Dimensions 


4 . Task  Orientation 

the  amount  of  emphasis  on  completing  planned 
activities  and  staying  on  the  subject  matter 

5 . Competition 

how  much  students  compete  with  each  other  for  grades 
and  recognition  and  how  hard  it  is  to  achieve  good  grades 

System  Maintenance  and  Change  Dimensions 


6 . Order  and 
Organization 

the  emphasis  on  students  behaving  in  an  orderly 
and  polite  manner  and  on  the  overall  organization 
of  assignments  and  classroom  activities 

7.  Rule  Clarity 

the  emphasis  on  establishing  and  following  a clear 
set  of  rules  and  on  students  knowing  what  the 
consequences  will  be  if  they  don’t  follow  them;  the 
extent  to  which  the  teacher  is  consistent  in  dealing 
with  student  who  break  rules 

8.  Teacher  Control 

how  strict  the  teacher  is  in  enforcing  the  rules,  the 
severity  of  punishment  for  rule  infractions,  and  how 
much  students  get  into  trouble  in  the  class 

9 . Innovation 

how  much  students  contribute  to  planning  classroom 
activities,  and  the  extent  to  which  the  teacher  uses  new 
techniques  and  encourages  creative  thinking 

*Moos  & Trickett  (1987)  Classroom  Environment  Scale  Manual,  p.  2. 
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Appendix  F 

Student's  Perception  of  Ability  Scale 


Domain: 

outcomes;  affective 

Origin: 

Edmonton,  Albata 

Purpose: 

to  measure  subject-specific  self-concept  in  elementary  school  children 

Description: 

70  items;  7 scores:  general  ability;  arithmetic;  school  satisfaction; 
reading/spelling;  penmanship/neatness;  confidence;  total 

Age/grade  range: 
Test  development/ 
publication: 

grades  2 to  6 
1977 

Author(s)/availability: 

Frederic  J.  Boersma,  James  W.  Ch^man; 
Psychometrics  Canada,  Ltd. 

Reliability: 

alpha:  (n=642;  .69  to  .92;  median  of  .82) 

test-retest:  (4  to  6 weeks;  n=603;  .71  to  .83;  median  of  .78) 

Validity: 

discriminant,  correlates  very  weakly  with  Piers-Harris  Self-Concept  Scale 
(-.029  to  .078)  (used  by  authors  as  claim  of  distinctiveness  of  academic 
self-concept  from  general  self-ctmcept) 

construct,  correlates  with  scores  from  repot  cards  (n=642;  most  coefficients 
in  the  range  of  20  to  .49;  only  school  satisfaction  subscale  below  .20); 
correlates  with  CTBS  scores  (n=389;  strongest  correlations  woe  those 
between  general  ability,  arithmetic  and  full  scale  of  SPAS  and  CTBS 
scores;  weakest  were  school  satisfaction  and  penmanship  subscales); 
correlates  with  measures  of  intelligence  (Canadian  Cognitive  Abilities 
Test;  Otis-Lennon;  WISC-R):  ail  coefficients  below  .30; 
learning  disabled  students  scored  significantly  lower  on  the  SPAS  Full 
Scale  than  normals  (3  studies;  t-tests;  p<.01) 
predictive:  scores  of  SPAS  at  beginning  of  school  year  with  scores  on 
report  cards  at  year-end  (n=293;  similar  pattern  of  correlations  as  with 
previous  study  of  642  students) 

Norms: 

642  students  in  grades  3 to  6;  collected  in  April/May  1977  from  two 

schools  in  Edmonton;  separate  norms  by  gender; 

authOTS  recommend  development  of  lock  ntxms  whenever  possible 

Review(s): 

not  yet  reviewed  by  MMY 

Strengths: 

confidence  intervals  provided  for  use  in  the  interpretation  of  change  over 
time;  use  of  factcx'  aiialy  tic  techniques  to  determine  subscales 

Weaknesses: 

norms  not  very  representative  (only  2 schools  involved);  unclear  if 
instrument  is  recommended  for  use  in  grade  2 (norms  collected  but  not 
used);  school  satisfaction  and  penmanship  subscales  show  weak  validity 

Usefulness  for  EQI: 

yes 

Reference: 

Boersma,  FJ.,  Chapman,  J.W.  & Maguire,  T.  (1979).  The  Student's  Perception  of  Ability  Scale:  an 
instrument  ftx*  measuring  academic  self-concept  in  elementary  school  children.  Educational  and 
Psychological  Measurement,  39,  1035-1041. 

Note.  From  Educational  Quality  Indicators:  Inventory  of  Assessment  Instruments  ( p.  60)  by  Alberta  Education, 
1990,  Edmonton:  Corporate  and  Fiscal  Planning.  Reprinted  by  permission. 
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Appendix  G 


Teacher  Performance  Baseline:  Behaviors  Addressed 

During  Year  Three  (1991-92) 


1.6  The  teacher  ensures  that  objectives  are  stated. 

(Responsibility:  Divisional  PD  activity) 

By  stating  the  purpose  of  the  lesson,  the  teacher  lets  the  students  know  what  they  will 
learn  and  how  they  indicate  what  has  been  learned  Research  clearly  indicates  that 
students’  academically  engaged  time  is  enhanced  considerably  when  objectives  are  known 
early  in  a lesson.  There  is,  in  turn,  a direct  relationship  between  the  amount  of  time 
students  are  actively  engaged  in  learning  and  the  amount  they  ultimately  learn  and 
retention  is  enhanced  through  the  provision  of  timely  review.  Professional  development 
in  the  area  of  lesson  preparation  (i.e.,  Madeline  Hunter  Model)  is  planned  during  the 
upcoming  year. 


1.9  The  teacher  plans  for  interdisciplinary  activities. 

(This  indicator  will  not  be  addressed  this  year.) 

Opportunities  for  cross-curricular  activities  are  numerous,  particularly  at  the  elementary 
level.  Although  this  behavior  is  not  being  formally  addressed  through  the  EQI  Project  this 
year,  considerable  time  and  energy  will  be  devoted  to  addressing  the  province’s  Program 
Continuity  initiatives  scheduled  to  be  in  place  by  September  of  1994. 


2.1  The  teacher  circulates  during  pupil  activities. 

(Responsibility:  Deputy  Superintendent) 

Circulating  throughout  the  room  is  considered  critical  to  effective  classroom  management 
It  allows  teachers  to  monitor  student  work  and  to  communicate  to  students  an  awareness  of 
their  behavior,  while  at  the  same  time  attending  to  their  academic  needs.  As  a preventive 
measure,  teacher  movement  seems  to  discourage  misbehavior.  Teacher  awareness  of  the 
need  to  circulate  will  be  addressed  through  a variety  of  initiatives  in  the  course  of  the  year. 


2.5  The  teacher  provides  a physical  environment  which  is  stimulating, 
enriched,  relevant  and  has  displays  of  student  work. 

(Responsibility:  Teacher  and  Division) 

Very  little  research  and  relevant  literature  exist  on  the  impact  of  facilities,  particularly 
aesthetic  attributes,  on  the  learning  process.  Researchers  do  propose,  however,  that  there 
is  a positive  difference  in  behavior  and  academic  attainment  in  schools  which  are  tidy  and 
clean,  in  good  decorative  condition,  and  which  display  students’  work  on  classroom  walls 
(Priftis,  1984).  All  but  one  of  the  divisional  schools  have  been  totally  renovated  within 
the  last  five  years.  The  exception.  Central  Peace  High  School,  has  recently  undergone  a 
major  face  lift.  Teachers  and  administrators  will  continue  to  focus  on  providing 
environments  conducive  to  learning. 
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2.6  The  teacher  exhibits  a variety  of  questioning  techniques. 

(Responsibility:  ATA  Professional  Development  Committee) 

One  of  the  goals  of  education  is  to  assist  students  to  develop  the  ability  to  think.  This  can 
be  promoted  through  asking  the  right  kinds  of  questions  (i.e..  Bloom’s  Taxonomy).  At 
the  division- wide  Professional  Development  Day,  all  teachers  will  be  participating  in  a 
half-day  workshop  on  effective  questioning  in  the  classroom  setting.  This  suggests  the 
significance  teachers  have  attached  to  questioning  techniques,  as  they  apply  to  learning. 


3.2  The  teacher  exhibits  a variety  of  positive  reinforcement  techniques. 
(Responsibility:  Deputy  Superintendent) 

Just  as  do  adults,  students  too  appreciate  recognition  for  effort  expended.  Praise  and 
encouragement,  given  under  appropriate  circumstances,  can  be  particularly  motivating.  Pt 
is  most  useful  for  the  uncertain  students  to  know  that  they  are  performing  the  assigned  task 
correctly.  Teacher  support  and  encouragement  lead  to  an  even  greater  effort.  Pn  Ae  course 
of  the  coming  year,  practical  and  proven  strategies  will  be  shar^  with  teachers. 


3.4  The  teacher  provides  timely  and  relevant  reviews. 

(Responsibility:  Divisional  PD  activity.  To  be  incorporated  into  1.6 

above) 
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Appendix  H 

Parent/Student  Survey  Results 

The  following  are  the  composite  numbers  for  each  of  the  three  annual  surveys.  Responses 
received  were  as  follows:  1990  - 81  students,  77  parents;  1991  - 234  students,  75 
parents;  1992  - 261  students,  75  parents. 


PARENT  RESPONSES 

STUDENT  RESPONSES 

1990  1991  1992 

1990  1991  1992 

(n=77)  (n=75)  (n=75) 

(n=81)  (n=234)  (n=261) 

l.Seauentiailv  Developed  Planning  and  Preparation 

1.1  The  teacher  develops  short  unit  and  long  term  flexible  plans  with  an 
evaluation  process. 


Very  Important 

50 

52 

63 

Very  Important 

52 

139 

177 

Less  Important 

21 

16 

7 

Less  Important 
Don’t  IQiow 

17 

62 

41 

Don’t  Kjiow 

6 

4 

6 

12 

33 

42 

No  Response 

0 

3 

1 

No  Response 

0 

0 

1 

1.2  The  teacher  adheres  to  Curriculum/Program  of  Studies 

documents. 

Very  Important 

39 

33 

50 

Very  Important 

29 

101 

183 

Less  Important 

33 

35 

24 

Less  Important 

41 

95 

48 

Don’t  Know 

5 

4 

0 

Don’t  IQiow 

11 

37 

28 

No  Response 

0 

3 

1 

No  Response 

0 

1 

2 

1.3  The  teacher  provides  a safe  and  organized  environment  with 
resources/equipment  readily  available. 

Very  Important 

72 

65 

69 

Very  Important 

55 

163 

212 

Less  Important 

3 

7 

5 

Less  Important 

20 

52 

46 

Don’t  IQiow 

2 

2 

0 

Don’t  IQiow 

6 

19 

3 

No  Response 

0 

1 

1 

No  Response 

0 

0 

0 

1.4  The  teacher  provides  for  individual  learning  needs  with  consideration 
of  student  abilities,  based  on  diagnostic  information. 

Very  Important 

72 

69 

70 

Very  Important 

60 

157 

214 

Less  Important 

2 

2 

4 

Less  Important 

18 

47 

26 

Don’t  Know 

2 

2 

0 

Don’t  IQiow 

3 

29 

21 

No  Response 

1 

2 

1 

No  Response 

0 

1 

0 
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PARENT  RESPONSES 


STUDENT  RESPONSES 


1990  1991  1992 

(n=77)  (n=75)  (n=75) 


1990  1991  1992 

(n=81)  (n=234)  (n=261) 


1.5  The  teacher  provides  a variety  of  learning  activities. 


Very  Important 

65 

65 

65 

Very  Important 

52 

164 

209 

Less  Important 

12 

8 

9 

Less  Important 

21 

49 

42 

Don't  IQiow 

0 

1 

0 

Don't  I^ow 

8 

20 

10 

No  Response 

0 

1 

1 

No  Response 

0 

1 

0 

1.6  The  teacher  ensures  that 

objectives  are  stated  and  understood. 

Very  Important 

70 

67 

67 

Very  Important 

60 

171 

222 

Less  Important 

6 

5 

7 

Less  Important 
Don't  Know 

13 

39 

36 

Don't  I^ow 

1 

0 

1 

8 

23 

3 

No  Response 

0 

3 

0 

No  Response 

0 

1 

0 

1.7  The  teacher 

provides  for  motivation  of  students. 

Very  Important 

58 

65 

65 

Very  Important 

50 

153 

191 

Less  Important 

16 

6 

7 

Less  Important 

23 

57 

56 

Don't  l6iow 

3 

2 

1 

Don't  IQiow 

8 

21 

14 

No  Response 

0 

2 

2 

No  Response 

0 

3 

0 

1.8  The  teacher  makes  learning 

activities  relevant 

Very  Important 

65 

68 

63 

Very  Important 

54 

171 

190 

Less  Important 

10 

6 

10 

Less  Important 

20 

46 

57 

Don't  IQiow 

2 

0 

2 

Don't  IQiow 

7 

17 

14 

No  Response 

0 

1 

0 

No  Response 

0 

0 

0 

1.9  The  teacher 

plans  for  interdisciplinary  activities. 

Very  Important 

47 

48 

46 

Very  Important 

43 

111 

136 

Less  Important 

22 

9 

23 

Less  Important 

25 

85 

98 

Don't  IQiow 

8 

16 

6 

Don't  IQiow 

13 

38 

27 

No  Response 

0 

2 

0 

No  Response 

0 

0 

0 

71 


PARENT  RESPONSES 

1990  1991  1992 

(n=77)  (n=75)  (n=75) 


STUDENT  RESPONSES 

1990  1991  1992 

(n=81)  (n=234)  (n=261) 


2.  Key  Instructional  Strategies 


2.1  The  teacher  circulates  during  pupil  activities. 


Very  Important 

50 

51 

61 

Very  Important 

22 

93 

115 

Less  Important 

23 

17 

14 

Less  Important 

47 

111 

125 

Don’t  I^ow 

4 

2 

0 

Don't  Kjiow 

12 

20 

21 

No  Response 

0 

5 

0 

No  Response 

0 

10 

0 

2.2  The  teacher 

presents  clear  assignments  which  are 

relevant  to  student 

experiences. 

Very  Important 

66 

71 

70 

Very  Important 

62 

176 

211 

Less  Important 

10 

3 

3 

Less  Important 

15 

41 

42 

Don't  Know 

1 

1 

2 

Don't  I^ow 

4 

17 

8 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

2.3  The  teacher  exhibits  classroom 

1 management  skills. 

Very  Important 

70 

69 

73 

Very  Important 

51 

143 

163 

Less  Important 

7 

2 

2 

Less  Important 

24 

70 

88 

Don't  IQiow 

0 

3 

0 

Don't  IQiow 

6 

20 

8 

No  Response 

0 

1 

0 

No  Response 

0 

1 

2 

2.4  The  teacher  provides  an  environment  whereby  active  learning  takes 

place  within  a working 

and  non-threatening  atmosphere. 

Very  Important 

72 

71 

70 

Very  Important 

62 

148 

192 

Less  Important 

5 

4 

5 

Less  Important 

13 

66 

57 

Don’t  IQiow 

0 

0 

0 

Don't  Know 

6 

20 

12 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

2.5  The  teacher  provides  a 

physical  classroom  environment  which  is 

stimulating,  enriched,  relevant  and 

has  displays  of  student  work. 

Very  Important 

65 

56 

65 

Very  Important 

37 

119 

146 

Less  Important 

10 

17 

9 

Less  Important 

32 

90 

no 

Don't  IQiow 

2 

2 

1 

Don't  K^ow 

12 

25 

5 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

2.6  The  teacher 

exhibits  a 

variety 

of  questioning  techniques. 

Very  Important 

60 

56 

60 

Very  Important 

37 

109 

127 

Less  Important 

15 

16 

12 

Less  Important 

32 

98 

116 

Don't  I^ow 

2 

3 

2 

Don't  Kjiow 

12 

27 

18 

No  Response 

0 

0 

1 

No  Response 

0 

0 

0 

72 


PARENT  RESPONSES 


STUDENT  RESPONSES 


1990  1991  1992 

(n=77)  (n=75)  (n=75) 


1990  1991  1992 

(n=81)  (n=234)  (n=261) 


3.  Communication  Skills 

3.1  The  teacher  provides  dear  explanation  of  objectives,  assignments  and 
behavioral  expectations. 


Very  Important 

74 

74 

73 

Very  Important 

65 

180 

221 

Less  Important 
Don’t  l6iow 

2 

1 

2 

Less  Important 

13 

42 

30 

1 

0 

0 

Don’t  IQiow 

3 

11 

10 

No  Response 

0 

0 

0 

No  Response 

0 

1 

0 

3.2  The  teacher  exhibits  a 

variety 

of  positive  reinforcement  techniques* 

Very  Important 

63 

66 

69 

Very  Important 

45 

137 

184 

Less  Important 

13 

7 

5 

Less  Important 
Don’t  Know 

29 

80 

70 

Don’t  lOiow 

1 

2 

1 

7 

16 

7 

No  Response 

0 

0 

0 

No  Response 

0 

1 

0 

3.3  The  teacher  is  an  active  listener. 

Very  Important 

75 

73 

73 

Very  Important 

46 

176 

236 

Less  Important 

2 

1 

2 

Less  Important 

27 

45 

18 

Don’t  IQiow 

0 

1 

0 

Don’t  Kjiow 

8 

12 

7 

No  Response 

0 

0 

0 

No  Response 

0 

1 

0 

3.4  The  teacher 

provides  timely  and  relevant  reviews. 

Very  Important 

56 

60 

69 

Very  Important 

46 

131 

175 

Less  Important 
Don’t  IQiow 

20 

14 

6 

Less  Important 

27 

77 

73 

1 

1 

0 

Don’t  I^ow 

8 

26 

13 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

3.5  The  teacher 

uses 

language  appropriate  to  the  developmental  level 

of  students. 

Very  Important 

69 

70 

67 

Very  Important 

55 

145 

200 

Less  Important 

7 

3 

6 

Less  Important 

18 

71 

56 

Don’t  Kjiow 

1 

2 

2 

Don’t  Know 

8 

18 

5 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

3.6  The  teacher 

maintains 

regular 

parental  information  flow. 

Very  Important 

62 

57 

67 

Very  Important 

13 

57 

65 

Less  Important 

15 

17 

7 

Less  Important 

49 

146 

175 

Don’t  I^ow 

0 

1 

1 

Don't  Know 

19 

31 

21 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

73 


PARENT  RESPONSES 


STUDENT  RESPONSES 


1990  1991  1992 

(n=77)  (n=75)  (n=75) 


1990  1991  1992 

(n=81)  (n=234)  (n=261) 


4.  Pupil/Teacher  Relationships 

4.1  Mutual  respect  is  exhibited  between  teacher  and  students. 


Very  Important 

76 

72 

73 

Very  Important 

66 

191 

227 

Less  Important 

0 

0 

2 

Less  Important 

13 

22 

23 

Don't  IQiow 

1 

0 

0 

Don't  IQiow 

2 

18 

11 

No  Response 

0 

3 

0 

No  Response 

0 

3 

0 

4.2  Clear  routines, 

expectations/rules  made  known 

& followed. 

Very  Important 

73 

68 

72 

Very  Important 

51 

153 

181 

Less  Important 
Don’t  l6iow 

3 

6 

3 

Less  Important 

25 

61 

72 

1 

1 

0 

Don't  IQiow 

5 

19 

8 

No  Response 

0 

0 

0 

No  Response 

0 

1 

0 

4.3  The  teacher  provides  for  positive  peer  interaction/support. 

Very  Important 

65 

60 

63 

Very  Important 

52 

162 

197 

Less  Important 

9 

12 

12 

Less  Important 

22 

57 

52 

Don't  IQiow 

3 

2 

0 

Don’t  l6iow 

7 

15 

12 

No  Response 

0 

1 

0 

No  Response 

0 

0 

0 

4.4  The  teacher  uses  a fair  and  meaningful  evaluation  process. 

Very  Important 

77 

74 

74 

Very  Important 

73 

192 

227 

Less  Important 
Don’t  l6iow 

0 

1 

1 

Less  Important 
Don't  lOiow 

6 

28 

28 

0 

0 

0 

2 

14 

5 

No  Response 

0 

0 

0 

No  Response 

0 

0 

1 

4.5  The  teacher 

provides  prompt  feedback  on  student  assignments. 

Very  Important 

61 

67 

67 

Very  Important 

50 

155 

212 

Less  Important 

15 

8 

8 

Less  Important 

25 

61 

45 

Don't  Know 

1 

0 

0 

Don't  IQiow 

6 

18 

4 

No  Response 

0 

0 

0 

No  Response 

0 

0 

0 

74 


